Information processing device, information processing method, recording medium storing program code, and biomedical-signal measuring system

ABSTRACT

An information processing device, an information processing method, a recording medium storing a program, and a biomedical-signal measuring system. The information processing device includes circuitry to obtain specific information of a first intensity distribution of a biomedical signal from a source, the specific information being displayed on a display, and extract, based on the specific information, a specified area of the first intensity distribution to obtain second intensity distribution of the biomedical signal to be superimposed on an image indicative of the source. The information processing method includes obtaining specific information of a first intensity distribution of a biomedical signal from a prescribed source, the specific information being displayed on a display, and extracting, based on the specific information, a specified area of the first intensity distribution to obtain second intensity distribution of the biomedical signal to be superimposed on an image indicative of the source.

CROSS-REFERENCE TO RELATED APPLICATION

This patent application is based on and claims priority pursuant to 35U.S.C. § 119(a) to Japanese Patent Application No. 2019-045904, filed onMar. 13, 2019, in the Japan Patent Office, the entire disclosure ofwhich is hereby incorporated by reference herein.

BACKGROUND Technical Field

Embodiments of the present disclosure relate to an informationprocessing device, an information processing method, a recording mediumstoring program code, and a biomedical-signal measuring system.

Background Art

When brain surgery or the like is to be performed, a target site that isan affected site of the brain to be removed and sites to be conservedwithout removal need to be specified. The sites to be conserved include,for example, the visual area, auditory area, somatosensory area, motorarea, and the language area of the brain. When some of such sites to beconserved is removed by mistake, the corresponding ability, including,for example, perception and movement, is impaired. For this reason,specifying a target site or sites to be conserved is crucial inperforming brain surgery or the like. In order to scan the brain foractivity in advance of such brain surgery or the like, physicalphenomena inside the brain are measured using, for example,magneto-encephalography, electro-encephalography (EEG), functionalMagnetic Resonance Imaging (fMRI), or functional near-infraredspectroscopy (fNIRS). Regarding the fMRI and fNIRS methods, biomedicalsignals are obtained by measuring the blood flow inside the brain.However, in view of the nature of such blood flow, the precision of thebrain-activity measurement is limited. By contrast,magneto-encephalography measures the magnetic field caused by theelectrical activity inside the brain, and the electro-encephalography(EEG) can measure the electrical activity inside the brain and obtainthe biomedical signals in waveform. Such biomedical signals undergo, forexample, time-frequency analysis and are displayed.

As such technologies to measure changes in brain activity using thetime-frequency analysis, technologies to analyze changes in brainactivity, which cannot be analyzed by averaging, are known in the art.

SUMMARY

Embodiments of the present disclosure described herein provide aninformation processing device, an information processing method, anon-transitory recording medium storing a program for causing a computerto execute the information processing method, and a biomedical-signalmeasuring system. The information processing device includes circuitryto obtain specific information of a first intensity distribution of abiomedical signal from a source, the specific information beingdisplayed on a display, and extract, based on the specific information,a specified area of the first intensity distribution to obtain secondintensity distribution of the biomedical signal to be superimposed on animage indicative of the source. The information processing methodincludes obtaining specific information of a first intensitydistribution of a biomedical signal from a prescribed source, thespecific information being displayed on a display, and extracting, basedon the specific information, a specified area of the first intensitydistribution to obtain second intensity distribution of the biomedicalsignal to be superimposed on an image indicative of the source.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of embodiments and the many attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings.

FIG. 1 is schematic diagram illustrating a biomedical-signal measuringsystem according to embodiments of the present disclosure.

FIG. 2 is a diagram illustrating a hardware configuration of aninformation processing device according to embodiments of the presentdisclosure.

FIG. 3 is a block diagram illustrating a functional configuration of aninformation processing device according to embodiments of the presentdisclosure.

FIG. 4 is a diagram illustrating a starting screen displayed on aninformation processing device, according to embodiments of the presentdisclosure.

FIG. 5 is a diagram illustrating a measurement and collection screenaccording to embodiments of the present disclosure.

FIG. 6 is a diagram illustrating a magnified view of an area of ameasurement and collection screen on the left side, according toembodiments of the present disclosure.

FIG. 7 is a diagram illustrating a magnified view of an area of ameasurement and collection screen on the right side, according toembodiments of the present disclosure.

FIG. 8 is a diagram illustrating a state immediately after an annotationis input, according to embodiments of the present disclosure.

FIG. 9 is a diagram illustrating an updated annotation list according toembodiments of the present disclosure.

FIG. 10 is a flowchart of the measurement and collection processesperformed by an information processing device, according to embodimentsof the present disclosure.

FIG. 11 is a diagram illustrating a time-frequency analysis screenaccording to embodiments of the present disclosure.

FIG. 12 is a diagram illustrating a heat map in which the range isexpressed in decibels, according to embodiments of the presentdisclosure.

FIG. 13 is a diagram illustrating a state where a specific position isdesignated on a heat map, according to embodiments of the presentdisclosure.

FIG. 14 is a diagram illustrating a state where three peaks areindicated on a heat map from a peak list, according to embodiments ofthe present disclosure.

FIG. 15 is a diagram illustrating a state where the display mode of eachpeak is changed on a heat map according to the data of each peak,according to embodiments of the present disclosure.

FIG. 16 is a diagram illustrating a state where a specific area isdesignated on a heat map, according to embodiments of the presentdisclosure.

FIG. 17 is a flowchart of region specifying processes according toembodiments of the present disclosure.

FIG. 18 is a diagram illustrating a state where one point is selected ona heat map, according to embodiments of the present disclosure.

FIG. 19 is a diagram illustrating a binarized image according toembodiments of the present disclosure.

FIG. 20 is a diagram illustrating how a continuous area is regarded as aspecified area, according to embodiments of the present disclosure.

FIG. 21A, FIG. 21B, and FIG. 21C are diagrams illustrating closingoperations, according to embodiments of the present disclosure.

FIG. 22 is a diagram illustrating how a rectangular area thatcircumscribes a continuous area is regarded as a specified area,according to embodiments of the present disclosure.

FIG. 23 is a diagram illustrating a state where a plurality of specificareas are designated on a heat map, according to embodiments of thepresent disclosure.

FIG. 24 is a diagram illustrating a state where anotherthree-dimensional image and three-view head image are added to atime-frequency analysis screen, according to embodiments of the presentdisclosure.

FIG. 25 is a diagram illustrating how a specified area is extracted froma predetermined area, according to embodiments of the presentdisclosure.

FIG. 26 is a diagram illustrating a three-dimensional image on atime-frequency analysis screen, according to embodiments of the presentdisclosure.

FIG. 27 is a diagram in which the state of the brain, which correspondsto the position designated on a heat map, is displayed in the center ona three-dimensional image, according to embodiments of the presentdisclosure.

FIG. 28 is a diagram in which the state of the brain, which correspondsto the range designated on a heat map, is displayed in the center on athree-dimensional image, according to embodiments of the presentdisclosure.

FIG. 29 is a diagram in which line segments are used to indicate to whattime and frequency on a heat map each one of the images of a braindisplayed as a three-dimensional image corresponds, according toembodiments of the present disclosure.

FIG. 30 is a diagram in which rectangular areas are used to indicate towhat time and frequency on a heat map each one of the images of a braindisplayed as a three-dimensional image corresponds, according toembodiments of the present disclosure.

FIG. 31A and FIG. 31B are diagrams illustrating how the display on athree-dimensional image and the display of the rectangular regions on aheat map move as the three-dimensional image is dragged, according toembodiments of the present disclosure.

FIG. 32A and FIG. 32B are diagrams illustrating how the display on athree-dimensional image and the display of the rectangular regions on aheat map move as one of the brain images on the three-dimensional imageis clicked, according to embodiments of the present disclosure.

FIG. 33A, FIG. 33B, and FIG. 33C are diagrams illustrating how theviewpoints of all brain images in the same row are changed when one ofthe viewpoints of the brain displayed on a three-dimensional image ischanged, according to embodiments of the present disclosure.

FIG. 34A, FIG. 34B, and FIG. 34C are diagrams illustrating how theviewpoints of all the brain images in all the rows are changed when oneof the viewpoints of the brain displayed on a three-dimensional image ischanged, according to embodiments of the present disclosure.

FIG. 35A, FIG. 35B, and FIG. 35C are diagrams illustrating in detail howthe viewpoint is changed in FIG. 34A, FIG. 34B, and FIG. 34C.

FIG. 36A, FIG. 36B, and FIG. 36C are another set of diagramsillustrating how the viewpoints of all the brain images in all the rowsare changed when one of the viewpoints of the brain displayed on athree-dimensional image is changed, according to embodiments of thepresent disclosure.

FIG. 37A, FIG. 37B, and FIG. 37C are diagrams illustrating the detailsof how the viewpoint is changed as in FIG. 36A, FIG. 36B, and FIG. 36C.

FIG. 38 is a diagram illustrating a state in which a comment is added toa three-dimensional image, according to embodiments of the presentdisclosure.

FIG. 39 is a diagram illustrating a three-view head image on atime-frequency analysis screen, according to embodiments of the presentdisclosure.

FIG. 40 is a diagram illustrating a cut model that is displayed as athree-dimensional image on a three-view head image, according toembodiments of the present disclosure.

FIG. 41 is a diagram illustrating the peak selected from a peak list ina three-view head image, according to embodiments of the presentdisclosure.

FIG. 42 is a diagram illustrating the peak selected from a peak list andthe peaks that are temporally close to each other around the selectedpeak, in a three-view head image, according to embodiments of thepresent disclosure.

FIG. 43 is a diagram illustrating a state in which the peak selectedfrom a peak list and the peaks that are temporally close to each otheraround the selected peak are indicated with varying colors, in athree-view head image, according to embodiments of the presentdisclosure.

FIG. 44 is a diagram illustrating a state in which a result of dipoleestimation is superimposed on the three-dimensional images on athree-view head image, according to embodiments of the presentdisclosure.

FIG. 45A, FIG. 45B, FIG. 45C, and FIG. 45D are diagrams eachillustrating a state in which a result of measuring a plurality ofobjects (heat map) is superimposed on the three-dimensional images of athree-view head image, according to embodiments of the presentdisclosure.

FIG. 46 is a diagram illustrating a state before the viewpoint ischanged for the three-dimensional images in a three-view head image,according to embodiments of the present disclosure.

FIG. 47 is a diagram illustrating a dialog box displayed when theviewpoint of the three-dimensional images in a three-view head image ischanged, according to embodiments of the present disclosure.

FIG. 48 is a diagram illustrating a setting in which the changes inviewpoint made on a three-dimensional image are applied to the viewpointof the three-dimensional images in the first row of three-dimensionalview, according to embodiments of the present disclosure.

FIG. 49 is a diagram illustrating a state in which the changes inviewpoint of a three-dimensional image in a three-view head image areapplied to the viewpoint of the three-dimensional images in the firstrow of three-dimensional view, according to embodiments of the presentdisclosure.

FIG. 50 is a diagram illustrating a setting in which the changes inviewpoint made on a three-dimensional image are reflected in thethree-dimensional images in the first and second rows ofthree-dimensional view, according to embodiments of the presentdisclosure.

FIG. 51 is a diagram illustrating a state in which the changes in theviewpoint of a three-dimensional image of a three-view head image arereflected in the first and second rows of three-dimensional view,according to embodiments of the present disclosure.

FIG. 52 is a diagram illustrating a setting in which the changes inviewpoint made on a three-dimensional image are symmetrically reflectedin the three-dimensional images in the first and second rows ofthree-dimensional view, according to embodiments of the presentdisclosure.

FIG. 53 is a diagram illustrating a state in which the changes in theviewpoint of a three-dimensional image of a three-view head image aresymmetrically reflected in the three-dimensional images in the first andsecond rows of three-dimensional view, according to embodiments of thepresent disclosure.

FIG. 54 is a diagram illustrating a setting in which newthree-dimensional images in which the changes in viewpoint made on athree-dimensional image are reflected are added to three-dimensionalview in a separate row, according to embodiments of the presentdisclosure.

FIG. 55 is a diagram illustrating a state in which new three-dimensionalimages in which the changes in viewpoint made on a three-dimensionalimage of a three-view head image are reflected are added tothree-dimensional view in a separate row, according to embodiments ofthe present disclosure.

FIG. 56 is a diagram illustrating the setting of a peak list, accordingto embodiments of the present disclosure.

FIG. 57 is a diagram illustrating a spatial peak according toembodiments of the present disclosure.

FIG. 58 is a diagram illustrating a peak in time and a peak infrequency, according to embodiments of the present disclosure.

FIG. 59 is a diagram illustrating how a specific peak is selected from adrop-down peak list, according to embodiments of the present disclosure.

FIG. 60 is a diagram illustrating a state in which the peak selectedfrom a pull-down peak list is reflected in a heat map, three-dimensionalview, and a three-view head image, according to embodiments of thepresent disclosure.

FIG. 61A and FIG. 61B are diagrams illustrating how the viewing of aheat map and a three-dimensional image are played back by operations ona replay control panel, according to embodiments of the presentdisclosure.

FIG. 62A and FIG. 62B are diagrams illustrating how the viewing of aheat map and a three-dimensional image are returned on a frame-by-framebasis by operations on a replay control panel, according to embodimentsof the present disclosure.

FIG. 63A and FIG. 63B are diagrams illustrating how the viewing of aheat map and a three-dimensional image are advanced on a frame-by-framebasis by operations on a replay control panel, according to embodimentsof the present disclosure.

FIG. 64 is a diagram illustrating from what viewpoint the images are tobe initially displayed with respect to a peak, according to embodimentsof the present disclosure.

FIG. 65 is a diagram illustrating from what viewpoint the images are tobe initially displayed with respect to a pair of peaks, according toembodiments of the present disclosure.

FIG. 66 is a diagram illustrating a state in which the images of thebrain viewed from the viewpoints as illustrated in FIG. 65 are displayedas the initial display in three-dimensional view.

FIG. 67A, FIG. 67B, FIG. 67C, and FIG. 67D are diagrams illustrating howa lumbar signal is transmitted to the upper side in chronological order,according to embodiments of the present disclosure.

The accompanying drawings are intended to depict embodiments of thepresent disclosure and should not be interpreted to limit the scopethereof. The accompanying drawings are not to be considered as drawn toscale unless explicitly noted.

DETAILED DESCRIPTION

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the presentdisclosure. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“includes” and/or “including”, when used in this specification, specifythe presence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

In describing example embodiments shown in the drawings, specificterminology is employed for the sake of clarity. However, the presentdisclosure is not intended to be limited to the specific terminology soselected and it is to be understood that each specific element includesall technical equivalents that have the same structure, operate in asimilar manner, and achieve a similar result.

In the following description, illustrative embodiments will be describedwith reference to acts and symbolic representations of operations (e.g.,in the form of flowcharts) that may be implemented as program modules orfunctional processes including routines, programs, objects, components,data structures, etc., that perform particular tasks or implementparticular abstract data types and may be implemented using existinghardware at existing network elements or control nodes. Such existinghardware may include one or more central processing units (CPUs),digital signal processors (DSPs), application-specific integratedcircuits (ASICs), field-programmable gate arrays (FPGAs), computers orthe like. These terms may be collectively referred to as processors.

Unless specifically stated otherwise, or as is apparent from thediscussion, terms such as “processing” or “computing” or “calculating”or “determining” or “displaying” or the like, refer to the action andprocesses of a computer system, or similar electronic computing device,that manipulates and transforms data represented as physical, electronicquantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

Some embodiments of an information processing device, an informationprocessing method, a non-transitory recording medium storing a program,and a biomedical-signal measuring system according to the presentdisclosure will be described below in detail with reference to thedrawings. Note that numerous additional modifications and variations arepossible in light of the above teachings. It is therefore to beunderstood that within the scope of the appended claims, the disclosureof the present disclosure may be practiced otherwise than asspecifically described herein. For example, elements and/or features ofdifferent illustrative embodiments may be combined with each otherand/or substituted for each other within the scope of this disclosureand appended claims.

A schematic configuration of the biomedical-signal measuring system 1according to the present embodiment is described with reference to FIG.1.

FIG. 1 is schematic diagram illustrating a biomedical-signal measuringsystem 1 according to embodiments of the present disclosure.

The biomedical-signal measuring system 1 (an example of an informationprocessing system) measures various kinds of biomedical signals of atest subject such as magneto-encephalography (MEG) signals andelectro-encephalography (EEG) signals, and displays the results ofmeasurement. The biomedical signals to be measured are not limited tothe magneto-encephalography (MEG) signals and electro-encephalography(EEG) signals as above, but may be, for example, any electrical signalthat is caused by cardiac activity (i.e., any electrical signal that canbe expressed in an electrocardiogram (ECG)). As illustrated in FIG. 1,the biomedical-signal measuring system 1 includes a measurement device 3that measures at least one biomedical signal of a test subject, a server40 that stores at least one biomedical signal measured by themeasurement device 3, and an information processing device 50 thatanalyzes at least one biomedical signal stored on the server 40. In thepresent embodiment, as illustrated in FIG. 1, the server 40 and theinformation processing device 50 are described as separate units.However, no limitation is indicated thereby. For example, at least someof the functions of the server 40 may be implemented by the informationprocessing device 50.

In the present embodiment as illustrated in FIG. 1, a test subject(person to be measured) lies on a measurement table 4 on his or her backwith electrodes (or sensors) attached to his or her head to measure theelectrical brain waves, and puts his or her head into a hollow 32 of aDewar 31 of the measurement device 3. The Dewar 31 is a container ofliquid helium that can be used at very low temperatures, and a number ofmagnetic sensors for measuring the brain magnetism are disposed on theinner surface of the hollow 32 of the Dewar 31. The measurement device 3collects the electrical signals and the magnetic signals through theelectrodes and the magnetic sensors, respectively, and outputs dataincluding the collected electrical signals and magnetic signals to theserver 40. Note that such collected electrical signals and magneticsignals may be referred to simply as “measurement data” in the followingdescription of the present embodiment. The measurement data recorded onthe server 40 is read and displayed by the information processing device50, and is analyzed by the information processing device 50. As known inthe art, the Dewar 31 equipped with magnetic sensors and the measurementtable 4 is inside a magnetically shielded room. However, for the sake ofexplanatory convenience, the illustration of such a magneticallyshielded room is omitted in FIG. 1.

The information processing device 50 synchronizes and displays thewaveform of the magnetic signals obtained through the multiple magneticsensors and the waveform of the electrical signals obtained through themultiple electrodes on the same time axis. The electrical signalsindicate the inter-electrode voltage value obtained for the electricalactivity of nerve cells (i.e., the flow of ionic charge caused at thedendrites of neurons during synaptic transmission). Moreover, themagnetic signals indicate minute changes in electric field caused by theelectrical activity of the brain. The magnetic field that is generatedby the brain is detected by a high-sensitivity superconducting quantuminterference device (SQUID). These electrical signals and magneticsignals are examples of biomedical signals.

FIG. 2 is a diagram illustrating a hardware configuration of theinformation processing device 50 according to the present embodiment.

A hardware configuration of the information processing device 50according to the present embodiment is described with reference to FIG.2.

As illustrated in FIG. 2, the information processing device 50 isprovided with a central processing unit (CPU) 101, a random accessmemory (RAM) 102, a read only memory (ROM) 103, an auxiliary memory 104,a network interface (I/F) 105, an input device 106, and a display device107, and these elements are interconnected through a bus 108.

The CPU 101 controls the entire operation of the information processingdevice 50, and performs various kinds of information processing.Moreover, the CPU 101 executes an information displaying program storedin the ROM 103 or the auxiliary memory 104, to control the display of ameasurement and collection screen 502 (see, for example, FIG. 5) and theanalyzing screen (see, for example, a time-frequency analysis screen 601in FIG. 11).

The RAM 102 is used as a work area of the CPU 101, and may be a volatilememory in which a desired control parameter or data are stored. The ROM103 is a nonvolatile memory in which a basic input and output program orthe like is stored. For example, the ROM 103 may store theabove-described information displaying program.

The auxiliary memory 104 may be, for example, a hard disk drive (HDD) ora solid state drive (SSD). The auxiliary memory 104 stores, for example,a control program to control the operation of the information processingdevice 50, various kinds of data used to operate the informationprocessing device 50, and files.

The network interface 105 is a communications interface used tocommunicate with a device such as the server 40 in the network. Forexample, the network interface 105 is implemented by a network interfacecard (NIC) that complies with the transmission control protocol(TCP)/Internet protocol (IP).

The input device 106 is, for example, a user interface such as a touchpanel, a keyboard, a mouse, and an operation key. The display device 107is a device for displaying various kinds of information thereon. Forexample, the display device 107 is implemented by the display functionof a touch panel, a liquid crystal display (LCD), or an organicelectroluminescence (EL). The measurement and collection screen 502 andthe analyzing screen 601 are displayed on the display device 107, andthe screen of the display device 107 is updated in response to input andoutput operation through the input device 106.

The hardware configuration of the information processing device 50 asillustrated in FIG. 2 is given by way of example, and different kinds ofdevices may further be provided. It is assumed that the informationprocessing device 50 as illustrated in FIG. 2 is configured by hardwaresuch as a personal computer (PC). However, no limitation is intendedthereby, and the information processing device 50 may be a mobile devicesuch as a tablet PC. In such a configuration, the network interface 105is satisfactory as long as it is a communication interface with radiocommunication capability.

FIG. 3 is a block diagram illustrating a functional configuration of theinformation processing device 50 according to the present embodiment.

As illustrated in FIG. 3, the information processing apparatus 50includes a collection and display controller 201, an analysis displaycontroller 202, a peak-list controller 203, a communication unit 204, asensor information acquisition unit 205, an analyzer 206, a storage unit207, and an input unit 208.

The collection and display controller 201 is a functional unit thatcontrols the visual display when the data output from a sensor is beingcollected, using methods as will be described below with reference toFIG. 5 to FIG. 10.

The analysis display controller 202 is a functional unit that controlsthe visual display of, for example, the signal strength of thebiomedical signal computed and obtained by the analyzer 206 based on thesensor data (electrical signals or magnetic signals) obtained by thesensor information acquisition unit 205, using methods as will bedescribed below with reference to FIG. 11 to FIG. 67. As illustrated inFIG. 3, the analysis display controller 202 includes a heat-map displaycontrol unit 211 (i.e., an example of a display controller), athree-dimensional display control unit 212 (i.e., an example of adisplay controller), a sectional-view control unit 213 (i.e., an exampleof a display controller), and a viewing control unit 214.

As will be described later in detail with reference to, for example,FIG. 11, the heat-map display control unit 211 is a functional unit thatcontrols the visual display of the heat map 611 of the time-frequencyanalysis screen 601. As illustrated in FIG. 3, the heat-map displaycontrol unit 211 includes an acquisition unit 221, a binarizing unit222, and an extraction unit 223.

The acquisition unit 221 is a functional unit that obtains the time andfrequency at the position on a heat map 611, as will be described laterin detail with reference to, for example, FIG. 11, specified by a userthrough the input unit 208 and the value of the signal strength of thebiomedical signal at the above-specified time and frequency. Each of theabove time and frequency specified by the user and the value of thesignal strength of the biomedical signal at the specified time andfrequency is an example of specific information.

The binarizing unit 222 is a functional unit that determines a thresholdbased on the value of the signal strength obtained by the acquisitionunit 221 and binarizes the image on the heat map 611, which will bedescribed later in detail, based on the determined threshold. In otherwords, the binarizing unit 222 performs binarization by, for example,setting 0 or 1 to the pixel value of each pixel on the heat map 611based on a result of the above determination of a threshold.

The extraction unit 223 is a functional unit that extracts, from thebinarized image obtained by the binarizing unit 222, an area thatincludes the pixels at the position indicated by the time and frequencyobtained by the acquisition unit 221, where the pixels having the samevalue (e.g., 1) are combined with each other.

The three-dimensional display control unit 212 is a functional unit thatcontrols the visual display of the three-dimensional view 612 of thetime-frequency analysis screen 601. The sectional-view control unit 213is a functional unit that controls the visual display of the three-viewhead image 613 on the time-frequency analysis screen 601. The viewingcontrol unit 214 is a functional unit that controls the viewing inaccordance with the operation of or input to a replay control panel 615on the time-frequency analysis screen 601.

The peak-list controller 203 is a functional unit that extracts a peakin signal strength that meets a specified condition and registers theextracted peak in a peak list 614 on the time-frequency analysis screen601, as will be described later in detail with reference to, forexample, FIG. 11.

The communication unit 204 is a functional unit that performs datacommunication with, for example, the measurement device 3 or the server40. The communication unit 204 is implemented by the network interface105 illustrated in FIG. 2.

The sensor information acquisition unit 205 is a functional unit toobtain sensor information (i.e., an electrical signal or magneticsignal) from the measurement device 3 or the server 40 through thecommunication unit 204. The analyzer 206 is a functional unit thatanalyzes the sensor data (measured and obtained signal) obtained by thesensor information acquisition unit 205 to compute and obtain a signalthat indicates the signal strength at various parts inside the brain(such a signal may also be referred to as a biomedical signal in thefollowing description).

The storage unit 207 is a functional unit that stores, for example, thedata of a biomedical signal that indicates the signal strength computedand obtained by the analyzer 206. The storage unit 207 is implemented bythe RAM 102 or the auxiliary memory 104 as illustrated in FIG. 2.

The input unit 208 is a functional unit that accepts an input operationof annotation to be added to the sensor information and various kinds ofinput operations for the time-frequency analysis screen 601. The inputunit 208 is implemented by the input device 106 as illustrated in FIG.2.

The collection and display controller 201, the analysis displaycontroller 202, the peak-list controller 203, the sensor informationacquisition unit 205, and the analyzer 206 as described above may beimplemented as the CPU 101 launches a program stored in a memory such asthe ROM 103 into the RAM 102 and executes the program. Note also thatsome of or all of the collection and display controller 201, theanalysis display controller 202, the peak-list controller 203, thesensor information acquisition unit 205, and the analyzer 206 may beimplemented by hardware circuitry such as a field-programmable gatearray (FPGA) or an application specific integrated circuit (ASIC), inplace of a software program.

The functional units as illustrated in FIG. 3 merely indicate functionsschematically, and no limitation is intended by such configurations. Forexample, a plurality of functional units that are illustrated asindependent functional units in FIG. 3 may be configured as a singlefunctional unit. Alternatively, the function of a single functional unitas illustrated in FIG. 3 may be divided into a plurality of functionsimplemented by a plurality of functional units.

FIG. 4 is a diagram illustrating a starting screen displayed on theinformation processing device 50, according to the present embodiment.The operations on the starting screen 501 are described below withreference to FIG. 4.

On the starting screen 501, selection keys “measurement and collection”and “analysis” are displayed. When the brain wave and brain magnetismare to be measured, in many cases, the person who measures and collectsthe data and the person who analyzes the data are different. Forexample, when the “measurement and collection” key is selected by ameasurement engineer (technician), the data measured by the measurementdevice 3 is sequentially stored on the server 40, and is read anddisplayed by the information processing device 50. On the other hand,when the “analysis” key is selected by a doctor after the measurementand collection is done, the recorded measurement data is read andanalyzed.

FIG. 5 is a diagram illustrating a measurement and collection screen 502according to the present embodiment.

As illustrated in FIG. 5, a measurement and collection screen 502includes an area 511 a on which the signal waveforms of measuredbiomedical signals (i.e., magnetic signals and electrical signals in thepresent embodiment) are displayed, and an area 511 b on which monitoringdata other than the signal waveform is displayed. The area 511 a onwhich signal waveform is displayed is arranged on the left side of thescreen when viewed from the technician, and the area 511B on whichmonitoring data other than the signal waveform is displayed is arrangedon the right side of the screen when viewed from the technician.Accordingly, there is an economy of motion between the movement of themouse from the area 511 a on the left side of the screen to the area 511b on the right side of the screen and the motion of the line of sight ofa technician that follows the movement of a waveform (detected in realtime and dynamically displayed from the left side of the screen to theright side of the screen), thereby providing improved efficiency.

In the area 511B of the display screen, a monitoring window 512 isdisplayed to monitor the state of a subject during measurement. Bydisplaying the live image of the subject while he/she is being measured,the reliability of the check and judgment of a signal waveform can beimproved as will be described later in detail. Note that FIG. 5illustrates a case in which the entirety of the measurement andcollection screen 502 is displayed on the display screen of a singlemonitoring display (i.e., the display device 107). However, nolimitation is indicated thereby, and the area 511 a on the left side ofthe screen and the area 511 b on the right side of the screen mayseparately be displayed by two or more monitoring displays.

FIG. 6 is a diagram illustrating a magnified view of an area of themeasurement and collection screen 502 on the left side, according to thepresent embodiment.

The area 511 a includes a first display area 530 in which the time dataof signal detection is displayed in the horizontal direction of thescreen, and second display areas 521 to 523 in which a plurality ofsignal waveforms based on the signal detection are displayed in parallelacross the screen.

In the example as illustrated in FIG. 6, the time data that is displayedin the first display area 530 is a time line including the timeindication given along a time axis 531. However, no limitation isindicated thereby, and such a time line may only be a band-like orbelt-like axis where no time (time in numbers) is displayed, or may onlybe the time (time in numbers) where no axis is given. Alternatively, aone time line may be displayed by displaying the time axis 531 under thesecond display area 523 in addition to the first display area 530 on thetopside of the screen.

In the area 511 a, a plurality of signal waveforms obtained by aplurality of similar kinds of sensors or various kinds of signalwaveforms obtained by a group of a plurality of different kinds ofsensors are displayed in a synchronous manner along the same time axis531. In the example as illustrated in FIG. 6, the waveforms of aplurality of magneto-encephalography (MEG) signals obtained from theright side of the head of a subject and the waveforms of a plurality ofmagneto-encephalography (MEG) signals obtained from the left side of thehead of a subject are displayed parallel to each other in the seconddisplay area 521 and the second display area 522, respectively. In thesecond display area 523, the waveforms of a plurality ofelectro-encephalography (EEG) signals are displayed in parallel. Thesewaveforms of a plurality of electro-encephalography (EEG) signalscorrespond to the voltage signals measured between pairs of electrodes.Each of these waveforms of a plurality of signals is displayed inassociation with the identification number or channel number of thesensor through which the signal is obtained.

Once measurement is started and the readings from each sensor arecollected, as time passes a signal waveform is displayed moving fromleft to right in each of the second display areas 521 to 523 in the area511 a. A vertical line 532 indicates the measurement time (presenttime), and moves from the left side to the right side of the screen.Once the signal waveform display reaches the right end of the area 511 a(i.e., until the right end of the time axis 531), the signal waveformgradually disappears from the left end of the screen to the right. Then,new signal waveforms are displayed at disappearing positions in sequencefrom the left side to the right side, and the line 532 also moves fromthe left end of the screen to the right. Together with the above changeson the display, the lapse of time is also displayed in the horizontalfirst display area 530 along the time axis 531 as the measurementprogresses. The measurement and collection continues until the stop key539 is touched or clicked.

In the present embodiment, when the technician (i.e., a person whocollects the data) notices, for example, irregularities in waveform anda singular point of amplitude on the signal waveform during the datarecording, he/she can mark a problematic point or area on the signalwaveform. The point or area of such a problematic point or area to bemarked can be specified by moving a mouse cursor or clicking with amouse. The specified point or area is highlighted on the signalwaveforms of the second display areas 521 to 523, and the specifiedresult is displayed along the time axis 531 of the first display area530 in a relevant point in time or time range. The marking informationincluding the display along the time axis 531 is stored together withthe signal waveform data. The specified point corresponds to particulartime, and the specified area corresponds to a certain area including theparticular time.

In the example illustrated in FIG. 6, an area including at least onechannel is specified at a time t1 in the second display area 523, andthe span of time including the time t1 is highlighted at the mark 523a-1. In association with the display of the mark 523 a-1, an annotation530 a-1 that indicates the result of specification is displayed at thecorresponding point in time in the first display area 530. At a time t2,another point in waveform or an area around that point is marked in thesecond display area 523, and a mark 523 a-2 is highlighted at that point(the time t2) or in the area around that point (the time t2) (where atleast one of a time range or a plurality of waveforms is indicated). Atthe same time, an annotation 530 a-2 is displayed at the correspondingpoint in time (time range) in the first display area 530. Note that theterm “annotation” indicates that related information is given to certaindata as an annotation. An annotation according to the present embodimentis displayed at least based on the specified time data in associationwith the position at which the waveform is displayed based on the timedata. When a plurality of channels are, the annotation according to thepresent embodiment displayed in association with the correspondingchannel information.

The annotation 530 a-1 that is added to the first display area 530 atthe time t1 includes, for example, an annotation identification numberand the waveform-attribute information. In the present embodiment, anicon that indicates the attributes of the waveform and the text datasaying “strong spike” are displayed together with the annotation number“1.”

Once the technician specifies another point in waveform or an areaaround that point in waveform at the time t2, the mark 523 a-2 ishighlighted at the specified point, and an annotation number “2” isdisplayed at the corresponding point in time in the first display area530. Further, a pop-up window 535 for selecting the attribute isdisplayed at the highlighted point. The pop-up window 535 includesselection keys 535 a for selecting the various kinds of attribute, andan input box 535 b through which a comment or additional information isinput. On the selection keys 535 a, the causes of irregularities inwaveform such as fast activity, eye motion, body motion, and spike areindicated as the attributes of waveform. As the technician can check thestate of the subject through the monitoring window 512 of the area 511 bin the screen, he/she can appropriately select the attribute indicatingthe causes of irregularities in waveform. For example, when a spikeoccurs in a waveform, the technician can determine whether such a spikeshows symptoms of epilepsy or caused by the body motion (such as asneeze) of the subject.

The same operations are also performed at the time t1. In FIG. 6, as theselection key 535 a of “spike” is selected in the pop-up window 535 and“strong spike” is input to the input box 535 b, the annotation 530 a-1is displayed in the first display area 530. Due to such a display mode,when a large number of signal waveforms are displayed along the sametime axis 531 in a synchronous manner, a point of interest or region ofinterest of the signal waveforms can visually be recognized andidentified easily, and the basic information at a point of interest caneasily be figured out.

Some of or all of the annotation 530 a-1, for example, at least one ofan attribute icon and a text data may be displayed in the proximity ofthe mark 523 a-1 on the signal waveforms in the second display area 523.When such an annotation is added directly over the signal waveforms, theability to check the shape of the waveforms may be impaired. For thisreason, when an annotation is displayed over the signal waveforms in thesecond display areas 521 to 523, it is desired that display ornon-display of such an annotation be selectable.

The counter box 538 displays the cumulative number of spike annotations.In the present embodiment, every time “spike” is selected, the countervalue in the counter box 538 is incremented. Accordingly, the analystcan instantly figure out the total number of spikes selected until now(as indicated by a line 532) since the recording has started.

FIG. 7 is a diagram illustrating a magnified view of an area of themeasurement and collection screen 502 on the right side, according tothe present embodiment.

In FIG. 7, a state at the same time as illustrated in FIG. 6 (the pointin time indicated by the line 532) is illustrated. In the monitoringwindow 512 of the area 511 b, the live image of a state in which asubject lies on the measurement table 4 and the head of the subject isinside the measurement device 3 is displayed. In the area 511 b, themagnetoencephalogram distribution maps 541 and 542, the brain-wavedistribution map 550, and the annotation list 560, each of whichcorresponds to one of the signal waveforms in the second display areas521, 522, and 523, are displayed. The annotation list 560 is a list ofannotations of the signal waveforms as illustrated in FIG. 6. Every timethe point or area on the signal waveforms is specified in the seconddisplay areas 521 to 523 and annotated, the associated information issequentially added to the annotation list 560. When information is addedto the annotation list 560 on the measurement and collection screen 502,such information is displayed, for example, in descending order wherenew data is displayed on an upper side). However, no limitation isintended thereby. For example, the annotation list 560 may be displayedin ascending order. The annotation list 560 is displayed such that therelation with the annotation displayed in the first display area 530along the time axis 531 will be clear to the analyst. Alternatively, thedisplay order may be changed, or information may be sorted according tothe type of item.

In the example as illustrated in FIG. 7, the time data that correspondto the annotation number “1” and the added annotation are listed in theannotation list 560. As the annotation, an attribute icon that indicates“spike” and the text saying “strong spike” are recorded. When the mark523 a-1 is highlighted, the time data that correspond to the annotationnumber “2” is listed. In the present embodiment, the term “annotation”may be considered to be a group of information including an annotationnumber, time data, and annotation, or may be considered to be only theannotation. Additionally, the term “annotation” may be considered to bea group of information including annotation and an annotation number ortime data.

A selection box 560 a to choose show/hide is arranged near theannotation list 560. When “hide” is selected in the selection box 560 a,the annotation other than a highlighting mark on the signal waveforms ishidden from view in the second display areas 521 to 523. However, thedisplay of the annotation in the first display area 530 along the timeaxis 531 is maintained. Due to such a configuration, the annotationbecomes recognizable without impairing the recognizability of signalwaveforms.

FIG. 8 is a diagram illustrating a state immediately after an annotationis input, according to the present embodiment.

More specifically, FIG. 8 illustrates a screen displayed immediatelyafter “spike” is selected from the pop-up window 535 at the time t2 anda text “normal spike” is input. When “OK” key is selected from thepop-up window 535 as illustrated in FIG. 6, the pop-up window 535 closesand an annotation 530 a-2 is displayed at the corresponding point intime in the first display area 530 as illustrated in FIG. 8. Inassociation with the annotation number “2,” an attribute icon thatindicates “spike” and text data saying “normal spike” are displayed. Atthe same time, the value in the counter box 538 is incremented.Moreover, an attribute icon 526-2 is displayed near the highlighted mark523 a-2. In the present embodiment, the attribute icon 526-1 is alsodisplayed near the mark 523 a-1. However, as described above, theattribute icons 526-1 and 526-2 may be displayed or hidden in aselective manner. The annotation includes annotation A1 including themark 523 a-1 and the attribute icon 526-1 and annotation A2 includingthe mark 523 a-2 and the attribute icon 526-2.

FIG. 9 is a diagram illustrating an updated annotation list according tothe present embodiment.

The annotation list 560 is updated as the annotation that corresponds tothe mark 523 a-2 is added to the area 511 a on the left side of themeasurement and collection screen 502. As a result, a memo saying“normal spike” is added to the annotation number “2.”

Every time a desired point or area on the signal waveforms is specifiedin the area 511 a during the measurement, the specified point ishighlighted, and the annotation is displayed in the first display area530 along the time axis 531. In the area 511 b, the annotation issequentially added to the annotation list 560.

It is not always necessary to display an annotation number in theannotation list 560 and the area 511 a where signal waveforms aredisplayed, and the display of an annotation number may be omitted. Anyinformation can be used as identification information as long as theadded annotation can be recognized by that information. For example, anattribute icon, attribute texts (e.g., “strong spike”), and time in theproximity of the time axis 531 may be displayed in association with eachother. Further, a file number (i.e., the number displayed in the item“File” as illustrated in FIG. 9) may be displayed along with the area511 a.

When the stop key 539 (see FIG. 8) is selected (touched or clicked) andthe measurement is terminated, the highlighted portion specified in thesecond display areas 521 to 523 is stored in association with the signalwaveform. The annotation that is displayed at the corresponding point intime in the first display area 530 is also stored in association withthe annotation number and the time. Relevant information such as thecounter value in the counter box 538 and the items in the annotationlist 560 is also stored. By storing the above display information, evenif the technician and the analyst are different, the analyst can easilyrecognize and analyze a problematic portion.

FIG. 10 is a flowchart of the measurement and collection processesperformed by the information processing device 50, according to thepresent embodiment.

The measurement and collection that is performed by the informationprocessing device 50 according to the present embodiment below withreference to FIG. 10.

When “measurement and collection” is selected on the starting screen 501as illustrated in FIG. 4 (step S11), the measurement is started, and thedisplay is controlled in a synchronous manner along a time axis wherethe waveforms of a plurality of signals are equivalent to each other(step S12). In the present embodiment, the term “a plurality of signalwaveforms” includes both the signal waveform detected by a plurality ofsensors of the same kind and the multiple signal waveforms detected by aplurality of various kinds of sensors. In the present embodiment, thewaveforms of biomedical signals consist of the waveform of the magneticsignals obtained through a plurality of magnetic sensors from the rightside of the head of a subject, the waveform of the magnetic signalsobtained through a plurality of magnetic sensors from the left side ofthe head of the subject, and the waveform of the electric signalsobtained through electrodes for measuring the electrical brain waves ofthe subject. However, no limitation is intended thereby. The sensors maybe selected not just between the right and left groups of sensors, butmay be selected from any part of the brain such as a parietal region, afrontal lobe, and a temporal lobe. When sensors at a parietal region areselected in “MEG Window Control 1” as illustrated in, for example, FIG.7, the sensors other than sensors at a parietal region are selected in“MEG Window Control 2.”

The information processing device 50 determines whether any designationis made as a point of interest or region of interest in the displayedsignal waveform (step S13). When such designation is made as a point ofinterest or a range of interest (YES in the step S13), the display iscontrolled to highlight the designated point in the display areas ofsignal waveform (i.e., the second display areas 521 to 523), and todisplay the results of selection in a relevant point in time of thetime-axis field (i.e., the first display area 530) (step S14). Theresult of designation includes data indicating that the designation hasbeen made or the identification information of the designation. Then,whether or not there is a request to input an annotation is determinedat the same time as when the results of designation are displayed in thetime-axis field or before or after the results of designation aredisplayed in the time-axis field (step S15). When there is a request toinput an annotation (YES in the step S15), the input annotation isdisplayed in a relevant point in time of the time-axis field, and theinput annotation is added to the annotation list so as to be displayedtherein (step S16). Then, whether or not a measurement terminationcommand has been input is determined (step S17). On the other hand, whenno point of interest or range of interest is designated (NO in the stepS13) and when there is no request to input an annotation (NO in the stepS15), the process proceeds to a step S17, and whether or not themeasurement is completed is determined. Steps S13 to S16 are repeateduntil the measurement is completed (YES in the S17).

Due to the above information displaying method, the measurement andcollection screen 502 can be provided in which the visibility of thesignal data is high when signals are collected from a plurality ofsensors.

FIG. 11 is a diagram illustrating a time-frequency analysis screen 601according to the present embodiment.

The analyzing operations that are performed on the time-frequencyanalysis screen 601, which is displayed on the information processingdevice 50, are described below with reference to FIG. 11.

When an “analysis” key is touched or clicked on the starting screen 501as described above with reference to FIG. 4, the analyzer 206 analyzesthe sensor information (i.e., an electrical signal or magnetic signal)that is collected by the above measurement and collection processes thatare performed on the measurement and collection screen 502, and computesand obtains a biomedical signal that indicates the signal strength atvarying points inside the brain (an example of a biological site or asource). As a method of calculating the signal strength, for example,spatial filtering is known in the art. However, no limitation isindicated thereby, and any other method may be adopted. When an“analysis” key is selected on the starting screen 501 as described abovewith reference to FIG. 4, the analysis display controller 202 controlsthe display device 107 to display the time-frequency analysis screen 601as illustrated in FIG. 11. As illustrated in FIG. 11, an analyzingscreen switching list 605, a heat map 611, a three-dimensional view 612,a three-view head image 613, a peak list 614, and a replay control panel615 are displayed the time-frequency analysis screen 601. An object ofthe analysis and measurement that is performed using the time-frequencyanalysis screen 601 is to mark and display critical sites of the brainfor human to live, such as a visual area, auditory area, somatosensoryarea, motor area, and a language area. A peak-list setting key 614 athat is displayed on the right side of the peak list 614 is used todisplay a window to configure the conditions for a peak to be registeredin the peak list 614. How the conditions for a peak to be registered inthe peak list 614 are configured by touching or clicking the peak-listsetting key 614 a will be described later in detail. The display andoperation of the heat map 611, the three-dimensional view 612, thethree-view head image 613, the peak list 614, and the replay controlpanel 615 will be described later in detail.

The analyzing screen switching list 605 is used to make a selection fromamong various kinds of analyzing screens. In addition to or in place ofthe time-frequency analysis screen 601 according to the presentembodiment where analysis is performed in regard to time and frequencybased on a biomedical signal, the analyzing screens selectable from theanalyzing screen switching list 605 may include, for example, ananalyzing screen where dipole estimation is performed to estimate oranalyze a site indicative of epilepsy or the like based on a biomedicalsignal. In the present embodiment, analyzing operations on thetime-frequency analysis screen 601 are described.

Some operations to be made on the heat map 611 of the time-frequencyanalysis screen 601 are described with reference to FIG. 12 to FIG. 15.

FIG. 12 is a diagram illustrating the heat map 611 in which the range isexpressed in decibel, according to the present embodiment.

FIG. 13 is a diagram illustrating a state where a specific position isdesignated on a heat map, according to the present embodiment.

FIG. 14 is a diagram illustrating a state where three peaks areindicated on a heat map from a peak list, according to the presentembodiment.

FIG. 15 is a diagram illustrating a state where the display mode of eachpeak is changed on a heat map according to the data of each peak,according to the present embodiment.

Time-frequency decomposition is performed on the biomedical signalscomputed and obtained by the analyzer 206, each of which indicates thesignal strength at a position inside the brain, and as illustrated inFIG. 11, the heat map 611 is an figure in which the horizontal axis andthe vertical axis indicate the time (i.e., the time elapsed since atriggering time) and the frequency, respectively, and thetwo-dimensional distribution (first intensity distribution) of thesignal strength of the biomedical signals, which is specified by thetime and frequency, is expressed by color. In other words, on the heatmap 611, colors are indicated on a pixel-by-pixel basis by pixel valuesindicative of colors (for example, red, green, and blue (RGB)) thatindicate the differences in signal strength. However, each pixel issubstantially associated with the signal strength of the biomedicalsignal. In the example as illustrated in FIG. 11, the signal strength isindicated by the variations with reference to, for example, a prescribedreference value. In the present embodiment, a prescribed reference valueis, for example, 0% as the average of the signal strength when nostimulus is given to a test subject.

In the present embodiment, illustration is made based on the premisethat the average of the signal strength varies between 0±100%. However,no limitation is intended thereby. When the average of the signalstrength varies beyond 1000%, the range in the illustration may bechanged to, for example, 200%. Alternatively, for example, decibels (dB)may be adopted in place of the percentage (%) as in the heat map 611 asillustrated in FIG. 12, which is a diagram illustrating a heat map inwhich the range is expressed in decibels, according to embodiments ofthe present disclosure. For example, when some sort of stimulation isgiven to the test subject (for example, physical shock is given to thetest subject, an arm of the test subject is moved, the test subject ismade to listen to some spoken words, or the test subject is made tolisten to a sound) at time 0 millisecond (ms), the heat map 611indicates, at the later time, the state of activity of the brain afterthat stimulation is given to the test subject, and indicates, at thetime earlier than the time 0 ms, the state of activity of the brainbefore that stimulation is given to the test subject.

The display operation on the heat map 611 is controlled by the heat-mapdisplay control unit 211.

As illustrated in FIG. 13, a desired position (point) on the heat map611 can be specified as the analyst performs an operation or input(clicking or tapping operation) to the input unit 208. As illustrated inFIG. 13, for example, the heat-map display control unit 211 controls thedisplay to display the specified position like the specified point 621.In FIG. 13, the specified point 621 is indicated by a white-coloredrectangle. However, no limitation is intended thereby, and the specifiedpoint 621 may be indicated in any other display modes.

On the heat map 611 as illustrated in FIG. 13, the position specified bythe operation of or input to the input unit 208 is indicated. However,no limitation is indicated thereby, and the heat map and some peakpositions at the time and frequency that the item of peak data selectedfrom among the peaks registered in the peak list 614 indicates may bedisplayed. For example, the top N peak positions with reference to thepeak selected from the peak list 614 may be displayed on the heat map611. FIG. 14 is a diagram illustrating an example in which the positionsof the top three peaks are indicated, according to the presentembodiment. How the peak positions are to be indicated may be determinedbased on the settings. For example, in addition to or in place of theabove setting, the settings may be switched between the setting in whichno peak is to be indicated or the setting in which peaks whose signalstrength is equal to or higher than M are indicated.

As illustrated in FIG. 15, the display mode of the multiple peaksdisplayed on the heat map 611 may be changed according to the attributeinformation of those peaks. FIG. 15 is a diagram illustrating an examplein which a number is given to each of the indicated peaks and the colorsof each portion in which a number is indicated are changed so as to bedifferent from each other, according to the present embodiment.

When a particular position is specified on the heat map 611 as describedabove, the distribution of the signal strength of the biomedical signalsof the time and frequency corresponding to the specified position isdisplayed as a heat map (second heat map). Note that this heat map(second heat map) is different from the heat map on the heat map 611. Asillustrated in FIG. 11, for example, the distribution of the signalstrength of the biomedical signals of the time and frequencycorresponding to the specified position is displayed like sites 712 a-1to 712 a-5 and 712 b-1 to 712 b-5 on the images of the brain in thethree-dimensional view 612 (i.e., an example of an image indicative of asource), and the distribution of the signal strength of the biomedicalsignals of the time and frequency corresponding to the specifiedposition is displayed like the sites 713 a-1, 713 a-2, 713 b, 713 c, and713 d on the images of the brain in the three-view head image 613 (i.e.,an example of an image indicative of a source). More specifically, thedistribution of the signal strength of the biomedical signals of thetime and frequency corresponding to the position specified on the heatmap 611 is displayed as a red-to-blue heat map.

How an area or region is specified on a heat map is described below withreference to FIG. 16 to FIG. 25.

FIG. 16 is a diagram illustrating a state where a specific area isdesignated on a heat map, according to the present embodiment.

FIG. 17 is a flowchart of region specifying processes according to thepresent embodiment

FIG. 18 is a diagram illustrating a state where one point is selected ona heat map, according to the present embodiment.

FIG. 19 is a diagram illustrating a binarized image according to thepresent embodiment.

FIG. 20 is a diagram illustrating how a continuous area is regarded as aspecified area, according to the present embodiment.

FIG. 21A, FIG. 21B, and FIG. 21C are diagrams illustrating closingoperations, according to the present embodiment.

FIG. 22 is a diagram illustrating how a rectangular area thatcircumscribes a continuous area is regarded as a specified area,according to the present embodiment.

FIG. 23 is a diagram illustrating a state where a plurality of specificareas are designated on a heat map, according to the present embodiment.

FIG. 24 is a diagram illustrating a state where anotherthree-dimensional image and three-view head image are added to thetime-frequency analysis screen 601, according to the present embodiment.

FIG. 25 is a diagram illustrating how a specified area is extracted froma predetermined area, according to the present embodiment.

In the above description made with reference to FIG. 13 to FIG. 15, as aparticular position (point) is specified on the heat map 611, thedistribution of the signal strength of the biomedical signals of thetime and frequency corresponding to the specified position is displayedon each one of the three-dimensional view 612 and the three-view headimage 613 as a heat map. Further, there are needs for displaying, as aheat map, the distribution of the signal strength of the biomedicalsignal that corresponds to the specified area when an area of specifictime and frequency is specified on the heat map 611. However, even if anarea such as a rectangular area can be specified as the analystperforms, for example, a dragging operation or a swiping operation onthe input unit 208, an undesired area outside the area to be observed asa heat map, which indicates the distribution of signal strength, tendsto be included. In other words, as illustrated in FIG. 16 for example,an area such as the specified area 622 where the signal strength of thebiomedical signal is strong on the heat map 611 needs to be specified insome cases. It is expected the specified area 622 will be an area of thetime and frequency where strong biomedical signals are generated due tothe same brain activity. However, it is very difficult and annoying forthe analyst to manually specify such an area like the specified area 622as illustrated in FIG. 16. In order to handle such a situation, in thepresent embodiment, operations are described in which an area such asthe specified area 622 as illustrated in FIG. 16 is automaticallyextracted. More specifically, processes as illustrated in FIG. 16 aredescribed along the flowchart in FIG. 17, in which the specified area622 is automatically extracted as an area that includes the specifiedpoint 621 and meets a prescribed condition once a specific position(point) on the heat map 611 is specified as a specified point 621 as theanalyst performs an operation or input (clicking or tapping operation)to the input unit 208. When the specified area 622 as illustrated inFIG. 16 is automatically extracted, the distribution of the averagesignal strength of the biomedical signals of the time and frequencyincluded in the specified area 622 is displayed as a heat map (secondintensity distribution), which is different from the heat map on theheat map 611. For example, as illustrated in FIG. 11, the distributionof the average signal strength of the biomedical signals of the time andfrequency included in the specified area 622 is displayed like sites 712a-1 to 712 a-5 and 712 b-1 to 712 b-5 on the images of the brain in thethree-dimensional view 612 (i.e., an example of an image indicative of asource), and the distribution of the average signal strength of thebiomedical signals of the time and frequency included in the specifiedarea 622 is displayed like the sites 713 a-1, 713 a-2, 713 b, 713 c, and713 d on the images of the brain in the three-view head image 613 (anexample of an image indicative of a source).

In a step S21, for example, the analyst performs an operation or input(clicking or tapping operation) to the input unit 208 to specify thespecified point 621 that indicates a desired point (pixel) on the heatmap 611 that is magnified for view as illustrated in FIG. 18. On theheat map as illustrated in FIG. 18, the signal strength of a biomedicalsignal is indicated by the pixel value of each pixel, and a pixel withdenser hatching indicates greater signal strength of a biomedicalsignal. In such a configuration, the acquisition unit 221 obtains thetime and frequency at the specified point 621 and the value of thesignal strength of the biomedical signal at the above-specified time andfrequency. Then, the process shifts to the processes in a step S22.

In a step S22, the binarizing unit 222 determines a threshold based onthe value of the signal strength obtained by the acquisition unit 221 soas to binarize the image on the heat map 611. For example, thebinarizing unit 222 generates a binarized image as illustrated in FIG.19 where the pixel value of the pixels whose value of the signalstrength is equal to or greater than the value of the signal strengthobtained by the acquisition unit 221 is changed to “1” (black) and thepixel value of the pixels whose value of the signal strength is lessthan the value of the signal strength obtained by the acquisition unit221 is changed to “0” (white).

In the above description, the binarizing unit 222 determines a thresholdbased on the value of the signal strength obtained by the acquisitionunit 221. However, no limitation is intended thereby. For example, thebinarizing unit 222 may determine a threshold based on whether the valueof the signal strength obtained by the acquisition unit 221 is equal toor greater than a predetermined rate (for example, 80%), and may sortthe values into “0” or “1” based on the determined threshold. As aresult, a binarized image is obtained. In such a configuration, it isdesired that the predetermined rate be easily changeable using a slideror the like. Alternatively, the binarizing unit 222 may determine athreshold using a predetermined rate (for example, 70%) of the greatestsignal strength of pixels searched around the position specified by thetime and frequency obtained by the acquisition unit 221, and may sortthe values into “0” or “1” based on the determined threshold. As aresult, a binarized image is obtained. The binarizing unit 222 maydetermine a threshold using a threshold of a fixed rate for the signalstrength, and may sort the values into “0” or “1” based on thedetermined threshold. As a result, a binarized image is obtained.However, in such a configuration, when the value of the signal strengthobtained by the acquisition unit 221 is 0, the binarizing unit 222 hasto carry out exception handling. For example, the binarizing unit 222has display a dialog box to encourage the user to specify a differenttime and frequency. Alternatively, the binarizing unit 222 may determinea threshold based on the plausibility, distribution, or the standarddeviation of the signal strength, which can separately be obtained bystatistical analysis, instead of the value of signal strength. In such aconfiguration, the binarizing unit 222 sorts the values into “0” or “1”based on the determined threshold. As a result, a binarized image can beobtained. In this configuration, the heat map 611 may display a heat mapof the plausibility, distribution, or the standard deviation instead ofa heat map of signal strength.

Then, the process shifts to step S23.

In a step S23, the extraction unit 223 extracts, from the binarizedimage obtained by the binarizing unit 222, the area that includes thepixels at the position indicated by the time and frequency obtained bythe acquisition unit 221, where the pixels having the pixel value “1”are combined with each other, as the specified area 622 as illustratedin FIG. 20.

According to the above region specifying processes in the steps S21 toS23, as illustrated in FIG. 16, the specified area 622 can automaticallybe extracted from the specified point 621 that is specified by theanalyst. In other words, the analyst can specify the specified area 622that has a complicated shape, in an automatic manner, just by specifyinga point on the heat map 611 with a clicking operation or the like.

Note also that the extraction unit 223 may extract a specified area uponperforming the closing operations as illustrated in FIG. 21A, FIG. 21B,and FIG. 21C on the binarized image obtained by the binarizing unit 222.For example, it is assumed that the image as illustrated in FIG. 21A isa binarized image obtained by the binarizing unit 222. In the closingoperations, first of all, the extraction unit 223 performs expansionprocesses on the binarized image as illustrated in FIG. 21A to obtain animage as illustrated in FIG. 21B. Then, the extraction unit 223 performserosion processes on the image as illustrated in FIG. 21B to obtain animage as illustrated in FIG. 21C. Due to such a configuration, theextraction unit 223 can combine two or more connected regions that havea small gap therebetween on the originally-obtained binarized image.Accordingly, even if the precision of threshold setting for thebinarizing unit 222 to perform binarization is insufficient, a specifiedarea can be extracted in an appropriate manner. Note also that theextraction unit 223 may perform opening operations to execute dilationprocesses in place of or in addition to the closing operations, aftererosion processes. Due to such a configuration, the extraction unit 223can exclude, for example, small connected regions that are not to beincluded in a specified area.

What is to be extracted by the extraction unit 223 is not limited to thespecified area 622 that has a complicated shape, as illustrated in FIG.20. For example, as illustrated in FIG. 22, the extraction unit 223 mayextract a rectangular area that circumscribes an area corresponding tothe specified area 622 as obtained above, as a specified area 622 a. Forpurposes of simplification, the extraction unit 223 may check the valuesof signal strength in the up-and-down and right-and-left directionsbased on the position designated on the heat map 611 by clickingoperation or the like, and may extract a rectangular area where thevalue of signal strength is at least more than the value obtained bymultiplying the value of the signal strength at the specified positionby a predetermined rate (for example, 80%), as the specified area 622 a.

As illustrated in FIG. 23, the analyst may perform additional operation(for example, new clicking or tapping operation) on the input unit 208.Due to this configuration, when a specified point 621 a is specified inaddition to the specified area 622 that has already been specified, aspecified area 623 can additionally be specified through the regionspecifying processes performed on the specified point 621 a as describedabove with reference to FIG. 17. In such a case, as illustrated in FIG.24, a three-dimensional view 612 a and a three-view head image 613 a aredisplayed as the three-dimensional image and three-view head image thatcorrespond to the newly-specified specified area 623, respectively.Then, the distribution of the average signal strength of the biomedicalsignals corresponding to the time and frequency included in thespecified area 623 is displayed on the brain images of thethree-dimensional view 612 and the three-view head image 613 a as a heatmap. Note that this heat map is different from the heat map on the heatmap 611. When the information about multiple specifying operations isreceived through the heat map 611, the three-dimensional view 612 andthe three-view head image 613 that correspond to the above informationabout specifying operations are displayed in descending order of time ofreceipt (in the direction from top to bottom). FIG. 24 illustrates anexample in which the specified area 622 is selected and then thespecified area 623 is selected. Due to such manner of presentation, theanalyst can easily and intuitively figure out the situation.Alternatively, when the information about multiple selecting operationsis received through the heat map 611, the three-dimensional view 612 andthe three-view head image 613 that correspond to the above informationabout specifying operations are displayed in ascending order of time ofreceipt (in the direction from bottom to top). In such a configuration,the three-dimensional view 612 and the three-view head image 613 thatcorrespond to the latest selected area are displayed directly below theheat map 611. Accordingly, the shift of the line of vision of theanalyzer to the heat map 611, the three-dimensional view 612, and thethree-view head image 613 can be reduced. When a plurality of points arespecified on the heat map 611, not only areas such as the specifiedareas 622 and 623 but also a plurality of points such as the specifiedpoint 621 may be specified. When a plurality of positions (points orareas) are specified on the heat map 611 as described above, themultiple distributions of the signal strength of the biomedical signalsof the time and frequency corresponding to the specified positions canbe compared with each other.

For example, the timing at which the analyst clicks or taps a desiredpoint of the heat map 611 may be used as a trigger, and the extractionunit 223 may use the threshold that is determined in advance by thebinarizing unit 222 to extract each one of all the connected regionsfrom the binarized image, as a specified area. In such a configuration,a heat map may be superimposed on each one of the three-dimensional view612, a three-dimensional view 612 a, and so on, and the three-view headimage 613, a three-view head image 613 a, and so on, which correspond toeach extracted specified area. Alternatively, a heat map thatcorresponds to the specified area that is selected by the analyst fromthe multiple specified areas extracted by the extraction unit 223 may bedisplayed on the three-view head image 613 and the three-dimensionalview 612. In such a configuration, immediately after a plurality ofspecified areas are extracted by the extraction unit 223, a heat mapthat corresponds to, for example, the specified area in which theaverage of the signal strength is largest may be displayed on thethree-view head image 613 and the three-dimensional view 612.

There are some cases in which areas of times and frequencies to befocused on are determined in advance on the heat map 611 according to aspecific human activity (such as hand shaking). For example, when thetarget human activity is movement, the analyst tends to perform analysisupon focusing on predetermined areas 625 to 627 in the heat map 611 asillustrated in FIG. 25. In such a configuration, the binarizing unit 222may binarize the areas 625 to 627, and may extract a combined area fromthe binarized image that corresponds to each of the binarized areas. Asa result, a specified area is determined. In such a configuration, theactual positions of the areas 625 to 627 are determined with referenceto, for example, the position of time 0 on the heat map 611. However,there are variations among individuals in the brain activity even if theareas 625 to 627 are analyzed in some extent. In view of suchcircumstances, it is desired that an imprecise area be set in advanceand the time and frequency be examined closely within that area. Forexample, the center point of each of the areas 625 to 627 (an example ofa representative point) is specified in advance, and assuming that theanalyst clicks or taps each of the center points, the specified areathat corresponds to each one of the center points can be extracted dueto the region specifying processes as described above with reference toFIG. 17. By so doing, a specified area can more appropriately beextracted taking in consideration the variations among individuals.

FIG. 26 is a diagram illustrating the three-dimensional view 612 on thetime-frequency analysis screen 601, according to the present embodiment.

FIG. 27 is a diagram in which the state of the brain, which correspondsto the position designated on the heat map 611, is displayed in thecenter on the three-dimensional view 612, according to the presentembodiment.

FIG. 28 is a diagram in which the state of the brain, which correspondsto the area designated on the heat map 611, is displayed in the centeron the three-dimensional view 612, according to the present embodiment.

FIG. 29 is a diagram in which line segments are used to indicate to whattime and frequency on the heat map 611 each one of the images of thebrain displayed as the three-dimensional view 612 corresponds, accordingto the present embodiment.

FIG. 30 is a diagram in which rectangular areas are used to indicate towhat time and frequency on the heat map 611 each one of the images ofthe brain displayed as the three-dimensional view 612 corresponds to,according to the present embodiment.

FIG. 31A and FIG. 31B are diagrams illustrating how the display on thethree-dimensional view 612 and the display of the rectangular regions onthe heat map 611 move as the three-dimensional view 612 is dragged,according to the present embodiment.

FIG. 32A and FIG. 32B are diagrams illustrating how the display on thethree-dimensional view 612 and the display of the rectangular regions onthe heat map 611 move as one of the brain images on thethree-dimensional view 612 is clicked, according to the presentembodiment.

Basic display operation of the three-dimensional view 612 on thetime-frequency analysis screen 601 is described below with reference toFIG. 26 to FIG. 32B.

As illustrated in FIG. 26, the three-dimensional view 612 is a view ofthe three-dimensional images (3D image) of the brain from a prescribedviewpoint, and the position (point or area) designated on the heat map611 or the signal strength of the biomedical signal that corresponds tothe peak selected from the peak list 614 is superimposed on thethree-dimensional view 612 as a heat map (second intensitydistribution). As illustrated in FIG. 26, in the same row of thethree-dimensional view 612, three-dimensional images of a brain from thesame viewpoint are displayed. In the example as illustrated in FIG. 26,the three-dimensional images of the brain as displayed in the displayarea 612-1 in the upper row of the three-dimensional view 612 are viewedfrom a viewpoint on the left side of the brain, and thethree-dimensional images of the brain as displayed in the display area612-2 in the lower row of the three-dimensional view 612 are viewed froma viewpoint on the right side of the brain. The display operation of thethree-dimensional view 612 is controlled by the three-dimensionaldisplay control unit 212.

As illustrated in FIG. 26, the three-dimensional view 612 consists ofthree-dimensional images of a brain viewed from two viewpoints, and suchthree-dimensional images of a brain are displayed in two rows. However,no limitation is intended thereby, and three-dimensional images of abrain may be displayed in any other numbers of rows. The number of rowsmay be changed as desired. For example, when the language area of thebrain is to be measured, the difference between right and left sides ofthe brain is crucial information. For this reason, the three-dimensionalimages of the brain that are viewed from two different viewpoints,consisting of a viewpoint on the right side of the brain and a viewpointon the left side of the brain, are to be displayed in two rows. Someexample associations between an object to be measured and desiredviewpoints are depicted in a first table given below. The object to bemeasured includes the stimulation given to a test subject during themeasurement (such stimulation is given by a stimulator and the rows inNo. 1 to No. 4 of the first table are relevant) and the motion made bythe test subject (see No. 5 of the first table), and indicates the itemsfrom which selection is to be made on the measurement and collectionscreen 502 when collection is to be performed. Once the object to bemeasured is selected, the three-dimensional view 612 of the brain thatis viewed from the corresponding viewpoint is displayed. The termviewpoint indicates the direction with the origin located at the frontof the test subject. As a matter of course, the number of rows may beedited in a separate manner. The three-dimensional view 612 asillustrated in FIG. 26 corresponds to No. 2 in the first table. For thesake of explanatory convenience, it is assumed in the followingdescription that the three-dimensional view 612 consists of two rows(images viewed from two viewpoints).

First Table OBJECT TO BE No. MEASURED VIEWPOINT 1 VISUAL REAR (VIEWPOINTTO VIEW OCCIPITAL PERCEPTION REGION FROM BACK OF OCCIPITAL REGION) ONEROW 2 AUDITORY RIGHT (VIEWPOINT TO VIEW RIGHT SENSATION TEMPORAL REGIONFROM OUTSIDE RIGHT TEMPORAL REGION) AND LEFT (VIEWPOINT TO VIEW LEFTTEMPORAL REGION FROM OUTSIDE LEFT TEMPORAL REGION) ONE ROW FOR RIGHT,ONE ROW FOR LEFT 3 LANGUAGE RIGHT AND LEFT 4 SOMATIC TOP SENSATION 5MOTION TOP

When the specified point 621 is specified on the heat map 611 asillustrated in FIG. 27, the three-dimensional display control unit 212sets the time that corresponds to the specified point 621 to the centerof the display area of the three-dimensional view 612, and controls thedisplay to display on the three-dimensional view 612 the heat map of thesignal strength on the brain at the times before and after the time thatcorresponds to the specified point 621. In the example as illustrated inFIG. 27, the time 560 ms is specified on the heat map 611. Accordingly,the intervals at which the brains are displayed are set to 5 ms, and theimages of the brain at 550, 555, 560, 565, and 570 ms around 560 ms aredisplayed on the three-dimensional view 612. However, no limitation isindicated thereby, and the intervals at which the images of the brainare displayed may be edited to, for example, 10 ms or 25 ms.

As illustrated in FIG. 28, when an area (i.e., the specified area 622)is specified on the heat map 611, the heat map of signal strength wherethe signal strength within that selected area is averaged may bedisplayed on the three-dimensional view 612. In such a configuration,the times of the neighboring three-dimensional images displayed on thethree-dimensional view 612 may be adjusted according to the selectedrange of time.

As illustrated in FIG. 28, for example, when the range of time of thespecified area 622 is between 450 and 600 ms and the intervals at whichthe neighboring three-dimensional images are displayed in thethree-dimensional view 612 is set to 150 ms, the range of time of thethree-dimensional image that is displayed in the center of thethree-dimensional view 612 is 450 to 600 ms. Moreover, the range of timeof the three-dimensional image on the left side of the three-dimensionalimage in the center of the three-dimensional view 612 is 300 to 450 ms,and the range of time of the three-dimensional image on the right of thethree-dimensional image in the center of the three-dimensional view 612is 600 to 750 ms. The heat map that is displayed on eachthree-dimensional image indicates, for example, the average in eachrange of time.

The association between the positions or ranges on the heat map 611 andthe multiple three-dimensional images on the three-dimensional view 612is described below with reference to FIG. 22 and FIG. 23. First of all,as illustrated in FIG. 22, when a specified point 621-1 is specified onthe heat map 611, the three-dimensional image of the brain, whichcorresponds to the time and frequency of the specified point 621-1, isdisplayed on the three-dimensional view 612. In this configuration, thethree-dimensional images of the brain at the times before and after thetime that corresponds to the specified point 621-1 are displayed aroundthe above three-dimensional image of the brain. In the example asillustrated in FIG. 22, the images of the brain that correspond to fivepoints in time are displayed. Accordingly, the heat-map display controlunit 211 control the display to display the points that correspond tothe respective points in time of the brain on the heat map 611 ascorresponding points 621-2 to 621-5, respectively. In such aconfiguration, the positions in frequency of the corresponding points621-2 to 621-5 are made consistent with the position in frequency of thespecified point 621-1. Further, as illustrated in FIG. 22, the heat-mapdisplay control unit 211 controls the display to display line segments631-1 to 631-5 that connect the specified point 621-1 and thecorresponding points 621-2 to 621-5 on the heat map 611 and thecorresponding three-dimensional images of the brain on thethree-dimensional view 612. Due to this configuration, to what positionson the heat map 611 the states of the brain as displayed on thethree-dimensional view 612 correspond to can be checked instantly. Inthe example as illustrated in FIG. 22, line segments are adopted.However, no limitation is indicated thereby, and any other ways ofassociation may be adopted. For example, the marks of the specifiedpoint 621-1 and the corresponding points 621-2 to 621-5 may beassociated with the colors of the background of the images of the brainin the three-dimensional view 612. In such a case, the specified point621-1 that is specified by the analyst is to be displayed in a modedistinguishable from the corresponding points 621-2 to 621-5.

When a specified area 622-1 is specified as a specific area on the heatmap 611, firstly, as illustrated in FIG. 23, the three-dimensional imageof the brain, which corresponds to the time and frequency on thespecified area 622-1, is displayed in the three-dimensional view 612. Inthis configuration, the specified area 622-1 is, for example, arectangular area that circumscribes the connected region extracted bythe extraction unit 223. In this configuration, the three-dimensionalimages of the brain at the ranges of time before and after the range oftime that corresponds to the specified area 622-1 are displayed aroundthe above three-dimensional image of the brain. In the example asillustrated in FIG. 30, the images of the brain that correspond to fiveranges of time are displayed. Accordingly, the heat-map display controlunit 211 control the display to display the ranges that correspond tothe respective ranges of time of the brain on the heat map 611 asrelated areas 622-2 to 622-5, respectively. In such a case, thespecified area 622-1 that is specified by the analyst is to be displayedin a mode distinguishable from the related areas 622-2 to 622-5. Forexample, the color of the rectangular frame of the specified area 622-1may be differentiated from the other frames. Further, as illustrated inFIG. 30, the three-dimensional display control unit 212 controls thedisplay to display rectangles similar to the specified area 622-1 andthe related areas 622-2 to 622-5 on the heat map 611 to surround thecorresponding three-dimensional images of the brain in thethree-dimensional view 612. Due to this configuration, to what ranges onthe heat map 611 the states of the brain as displayed in thethree-dimensional view 612 correspond to can be checked instantly. Whenthe specified area 622-1 is specified as a specific area on the heat map611, the frames of the specified area 622-1 and the related areas 622-2to 622-5 may be displayed, and frames 722-1 to 722-5 and the heat mapsmay be displayed in the three-dimensional view 612.

Operations where the display in the three-dimensional view 612 is movedto the right and left sides when a dragging operation, swipingoperation, or a cursor-movement key operation is performed in thethree-dimensional view 612 are described below with reference to FIG.31A, FIG. 31B, FIG. 32A, and FIG. 32B. FIG. 31A and FIG. 31B arediagrams illustrating a state in which the three-dimensional images inthe three-dimensional view 612 are moved to the right side by a draggingoperation, swiping operation, or a cursor-movement key operationperformed in the three-dimensional view 612, according to the presentembodiment. In such a case, as illustrated in FIG. 31A and FIG. 31B, asa result of the movement, the display of time is updated in accordancewith the brains that are currently displayed, and a rectangle isdisplayed to indicate that the three-dimensional image of the braindisplayed in the center of the three-dimensional view 612 is selected.Further, the three-dimensional display control unit 212 moves thedisplay of the specified area 622-1 and the related areas 622-2 to 622-5on the heat map 611 in accordance with the movement of thethree-dimensional images in the three-dimensional view 612.

As illustrated in FIG. 32A and FIG. 32B, when one of thethree-dimensional images other than the three-dimensional image in thecenter of the three-dimensional view 612 is clicked or tapped by theanalyst, the operated three-dimensional image of the brain moves to thecenter of the three-dimensional view 612. In the actual implementation,only the overlapping heat map may be moved and the positions of theimages of the brain may remain the same. In such a case, as illustratedin FIG. 32A and FIG. 32B, as a result of the movement, the display oftime is updated in accordance with the brains that are currentlydisplayed, and a rectangle is displayed to indicate that thethree-dimensional image of the brain displayed in the center of thethree-dimensional view 612 is selected. Further, the three-dimensionaldisplay control unit 212 moves the display of the specified area 622-1and the related areas 622-2 to 622-5 on the heat map 611 in accordancewith the movement of the three-dimensional images in thethree-dimensional view 612.

As described above, the display in the three-dimensional view 612 can bemoved as desired. Due to such a configuration, the changes in the stateof the brain across the time can quickly be recognized.

Operations in which the viewpoint of a desired three-dimensional imageof the three-dimensional view 612 on the time-frequency analysis screen601 is changed are described below with reference to FIG. 33A to FIG.38.

FIG. 33A, FIG. 33B, and FIG. 33C are diagrams illustrating how theviewpoints of all brain images in same row are changed when one of theviewpoints of the brain displayed in the three-dimensional view 612 ischanged, according to the present embodiment.

FIG. 34A, FIG. 34B, and FIG. 34C are diagrams illustrating how theviewpoints of all the brain images in all the rows are changed when oneof the viewpoints of the brain displayed in the three-dimensional view612 is changed, according to the present embodiment.

FIG. 35A, FIG. 35B, and FIG. 35C are diagrams illustrating in detail howthe viewpoint is changed in FIG. 34A, FIG. 34B, and FIG. 34C, accordingto the present embodiment.

FIG. 36A, FIG. 36B, and FIG. 36C are another set of diagramsillustrating how the viewpoints of all the brain images in all the rowsare changed when one of the viewpoints of the brain displayed in thethree-dimensional view 612 is changed, according to the presentembodiment.

FIG. 37A, FIG. 37B, and FIG. 37C are diagrams illustrating the detailsof how the viewpoint is changed as in FIG. 36A, FIG. 36B, and FIG. 36C,according to the present embodiment.

FIG. 38 is a diagram illustrating a state in which a comment is added tothe three-dimensional view 612, according to the present embodiment.

The viewpoint of the brains that are displayed as three-dimensionalimages in the three-dimensional view 612 can be changed as manipulatedby the analyst (for example, a dragging operation or a swipingoperation). Some patterns of a method of reflecting the changes made inthe viewpoint of a specific three-dimensional image of the brain in thethree-dimensional view 612, in the other three-dimensional images, aredescribed below.

Firstly, cases in which the viewpoint of the other three-dimensionalimages in the same row is changed in a similar manner when the viewpointof a specific three-dimensional images is changed are described. Asillustrated in FIG. 33A, it is assumed that the analyst has performed anoperation on the three-dimensional view 612 displayed in two rows tochange the viewpoint of the three-dimensional image at the right end(such a three-dimensional image may be referred to as a targetthree-dimensional image in the following description) from among themultiple three-dimensional images of the brain as displayed in thedisplay area 612-1. In such a case, as illustrated in FIG. 33B, asmanipulated by the analyst, the three-dimensional display control unit212 changes the viewpoint of the target three-dimensional images withthe viewpoint from the left side of the brain so as to display thethree-dimensional images of the brain viewed from a rear side. In sodoing, the viewpoint of the heat maps that are superimposed on theimages of the brain is also changed in a similar manner. Then, asillustrated in FIG. 33C, the three-dimensional display control unit 212changes the viewpoint of the other three-dimensional images of the brainin the same row as the target three-dimensional images (in the displayarea 612-1) in a similar manner to the target three-dimensional image.Due to this configuration, the changes that are made in the viewpoint ofa specific three-dimensional image (i.e., the target three-dimensionalimage) are automatically reflected in the other three-dimensional imagesin the same row. Accordingly, the operability or efficiency improves,and the changes in activity among the images of the brain that viewedfrom the same viewpoint and are temporally close to each other caneasily be checked. When the analyst wishes to change the viewpoint of athree-dimensional image, for example, he or she may manipulate mouse tomove the cursor onto the three-dimensional image whose viewpoint is tobe changed, and may perform, for example, dragging or clickingoperation. Alternatively, the analyst may designate a parameter in apop-up window.

Secondly, cases in which the viewpoint of the other three-dimensionalimages is changed in a similar manner when the viewpoint of a specificthree-dimensional images is changed are described. As illustrated inFIG. 34A, it is assumed that the analyst has performed an operation onthe three-dimensional view 612 displayed in two rows to change theviewpoint of the target three-dimensional image at the right end fromamong the multiple three-dimensional images of the brain as displayed inthe display area 612-1. In such a case, as illustrated in FIG. 34B, asmanipulated by the analyst, the three-dimensional display control unit212 changes the viewpoint of the target three-dimensional images withthe viewpoint from the left side of the brain so as to display thethree-dimensional images of the brain viewed from a rear side. Then, asillustrated in FIG. 34C, the three-dimensional display control unit 212changes the viewpoint of the other three-dimensional images of the brainin the same row as the target three-dimensional images (in the displayarea 612-1) in a similar manner to the target three-dimensional image.In other words, the three-dimensional display control unit 212 changesthe viewpoint of the other three-dimensional images of the brain asdisplayed in the display area 612-1 on the left side of the brain asillustrated in FIG. 28A so as to display the three-dimensional images ofthe brain viewed from a rear side, as illustrated in FIG. 28B. Further,as illustrated in FIG. 34C, the three-dimensional display control unit212 changes the viewpoint of the three-dimensional images in the otherrow (display area 612-2) different from the row of the targetthree-dimensional image, in a similar manner to the targetthree-dimensional image. In other words, as illustrated in FIG. 28C, thethree-dimensional display control unit 212 changes the viewpoint of thethree-dimensional images of the brain as displayed in the display area612-2 on the right side of the brain as illustrated in FIG. 28A, so asto display the three-dimensional images of the brain viewed from a frontside. If the processing capability is well above the actual load, theprocesses in FIG. 28A to FIG. 28C may be performed at high speed, as ifthe viewpoints of all the images of the brain appear to change at thesame time. On the other hand, if the processing capability is poor, theviewpoints of the other images may be changed when the change inviewpoint is determined (i.e., the timing at which the user releases akey of the mouse when the viewpoint is changed, for example, by rotatingthe image of the brain by dragging operation) after only the viewpointof the image that is moved by the user is changed. In so doing, therespective viewpoints of the heat maps that are superimposed on theimages of the brain are also changed in a similar manner. Due to thisconfiguration, the changes that are made in the viewpoint of a specificthree-dimensional image (i.e., the target three-dimensional image) areautomatically reflected in the other three-dimensional images in thesame row and the other rows. Accordingly, the operability or efficiencyimproves, and the changes in activity among the images of the brain thatare temporally close to each other can easily be checked.

Furthermore, cases are described in which, when the viewpoint of aspecific three-dimensional images is changed, the viewpoint of the otherthree-dimensional images in the same row is changed in a similar mannerand the viewpoint of the three-dimensional images in the other row ischanged in a corresponding manner. More specifically, the viewpoint ischanged to be symmetrical to the center plane of the brain (symmetryplane). As illustrated in FIG. 36A, it is assumed that the analyst hasperformed an operation on the three-dimensional view 612 displayed intwo rows to change the viewpoint of the target three-dimensional imageat the right end from among the multiple three-dimensional images of thebrain as displayed in the display area 612-1. In such a case, asillustrated in FIG. 36B, as manipulated by the analyst, thethree-dimensional display control unit 212 changes the viewpoint of thetarget three-dimensional images with the viewpoint from the left side ofthe brain, so as to display the three-dimensional images of the brainviewed from a left-frontal side. Then, as illustrated in FIG. 36C, thethree-dimensional display control unit 212 changes the viewpoint of theother three-dimensional images of the brain in the same row as thetarget three-dimensional images (in the display area 612-1) in a similarmanner to the target three-dimensional image. In other words, thethree-dimensional display control unit 212 changes the viewpoint of theother three-dimensional images of the brain as displayed in the displayarea 612-1 on the left side of the brain as illustrated in FIG. 37A soas to display the three-dimensional images of the brain viewed from aleft-frontal side, as illustrated in FIG. 37B. Further, as illustratedin FIG. 36C, the three-dimensional display control unit 212 changes theviewpoint of the three-dimensional images in the other row (display area612-2) different from the row of the target three-dimensional image, ina corresponding manner to the target three-dimensional image. In otherwords, the three-dimensional display control unit 212 changes theviewpoint of the three-dimensional images of the brain as displayed inthe display area 612-2 on the right side of the brain as illustrated inFIG. 37A to be symmetrical to the center plane of the brain (symmetryplane) as illustrated in FIG. 37C. In other words, the three-dimensionaldisplay control unit 212 changes the viewpoint so as to display thethree-dimensional images of the brain viewed from a right-frontal sideof the brain. In so doing, the respective viewpoints of the heat mapsthat are superimposed on the images of the brain are also changed in asimilar manner. Due to this configuration, the changes that are made inthe viewpoint of a specific three-dimensional image (i.e., the targetthree-dimensional image) are automatically reflected in the otherthree-dimensional images in the same row. Moreover, correspondingchanges in viewpoint are reflected in the three-dimensional images inthe other rows. Accordingly, the operability or efficiency improves.Furthermore, the images of the brain in multiple rows can be comparedwith each other, and thus the changes in activity among the images ofthe brain that are viewed from a corresponding viewpoint and aretemporally close to each other can be checked.

Any one of the three methods of reflecting changes in otherthree-dimensional images as described above may be adopted, or which oneof these methods is to be adopted to reflect changes may be switched byediting the settings.

As described above with reference to FIG. 33A to FIG. 37C, the firsttarget three-dimensional image that is to be manipulated by the analystto change its viewpoint in the present embodiment is thethree-dimensional image at the right end of the display area 612-1.However, no limitation is intended thereby, and any one of thethree-dimensional images in the display area 612-1 or the display area612-2 may be operated. A group of three-dimensional images included inthe display area 612-1 and a group of three-dimensional images includedin the display area 612-2 correspond to shape images and third imagesaccording to the present embodiment, respectively.

In the above description, the viewpoint of a specific three-dimensionalimage of the brain in the three-dimensional view 612 is changed, andoperations in which such a change in viewpoint is reflected in otherthree-dimensional images are described. However, no limitation isindicated thereby, and the display mode that is to be changed for athree-dimensional image is not limited to viewpoint. For example, thedisplay mode that is to be changed for a three-dimensional image may be,for example, changes in size, changes in brightness, or changes intransparency. Such changes may be reflected in other three-dimensionalimages without departing from the spirit or scope of the disclosure ofthe above changes in viewpoint.

After some changes are made on the three-dimensional images in thethree-dimensional view 612 as described above, as illustrated in FIG.38, the analyst may operate the input unit 208 to add a memo (forexample, a comment 635 as depicted in FIG. 38) onto a specificthree-dimensional image. Due to such a configuration, comments on anactive site of the brain that the analyst (for example, a doctor) isconcerned about can be recorded in association with the relevantthree-dimensional image, and can be applied to, for example,neurosurgery or a conference on such disorder of the brain.

Basic display operation of the three-view head image 613 on thetime-frequency analysis screen 601 is described below with reference toFIG. 39 to FIG. 45D.

FIG. 39 is a diagram illustrating the three-view head image 613 on thetime-frequency analysis screen 601, according to the present embodiment.

FIG. 40 is a diagram illustrating a cut model that is displayed as athree-dimensional image on the three-view head image 613, according tothe present embodiment.

FIG. 41 is a diagram illustrating the peak selected from the peak list614 in the three-view head image 613, according to the presentembodiment.

FIG. 42 is a diagram illustrating the peak selected from the peak list614 and the peaks that are temporally close to each other around theselected peak, in the three-view head image 613, according to thepresent embodiment.

FIG. 43 is a diagram illustrating a state in which the peak selectedfrom the peak list 614 and the peaks that are temporally close to eachother around the selected peak are indicated with varying colors, in thethree-view head image 613, according to the present embodiment.

FIG. 44 is a diagram illustrating a state in which a result of dipoleestimation is superimposed on the three-dimensional image 644 of thethree-view head image 613, according to the present embodiment.

FIG. 45A, FIG. 45B, FIG. 45C, and FIG. 45D are diagrams eachillustrating a state in which a result of measuring a plurality ofobjects (heat map) is superimposed on the three-dimensional image 644 ofthe three-view head image 613, according to the present embodiment.

As illustrated in FIG. 39, the three-view head image 613 includes thethree-dimensional image 644 and three sectional views viewed from adesired point of the brain from three directions (such three sectionalviews may be collectively referred to as a three-view image in thefollowing description). In the example as illustrated in FIG. 39, thethree-view head image 613 includes a sectional view 641 orthogonal tothe forward and backward directions of the brain, a sectional view 642orthogonal to the right and left directions of the brain, and asectional view 643 orthogonal to the up-and-down directions of the brainas the three sectional views viewed from a desired point of the brain inthree directions. In the sectional view 641, a reference line 645 a anda reference line 645 b that pass through the above-desired point aredrawn. In the sectional view 642, the reference line 645 a and areference line 645 c that pass through the above-desired point aredrawn. In the sectional view 643, the reference line 645 b and areference line 645 d that pass through the above-desired point aredrawn. A heat map that indicates the distribution of the signal strengthof the biomedical signal of the time and frequency that correspond tothe position (point or area) designated on the heat map 611, which isdifferent from the heat map 611, is superimposed on each one of thesectional views 641 to 643. The display operation on the three-view headimage 613 is controlled by the sectional-view control unit 213.

The reference line 645 a defines the position in the up-and-downdirections with reference to the above-desired point of the brain, andthus is drawn as a continuous line across the sectional view 641 and thesectional view 642. The reference line 645 b defines the position in theright and left directions with reference to the above-desired point ofthe brain, and thus is drawn as a continuous line across the sectionalview 641 and the sectional view 643. On the sectional view 642, thereference line 645 c defines the position in the forward and backwarddirections with reference to the above-desired point of the brain. Onthe sectional view 643, the reference line 645 d defines the position inthe forward and backward directions with reference to the above-desiredpoint of the brain. The sectional views 641 to 643 in the three-viewhead image 613 are arranged as above as illustrated in FIG. 39 becausethe reference line 645 a and the reference line 645 b can be drawn in acontinuous manner across a plurality of sectional views. However, nolimitation is intended thereby, and the sectional views 641 to 643 maybe arranged in any desired manner. In such a configuration, a referenceline that passing through a desired point of the brain may be drawn ineach one of the sectional views. Alternatively, no reference line may bedrawn in the sectional views. In such a configuration, for example, amark that indicates the desired point of the brain may be displayed oneach one of the sectional views.

The three-dimensional image 644 is a three-dimensional image of thebrain, and as will be described later, the viewpoints of thethree-dimensional images of the brain that are drawn in thethree-dimensional view 612 are changed in accordance with the operationmade to the three-dimensional image 644. A heat map that indicates thedistribution of the signal strength of the biomedical signal of the timeand frequency that correspond to the position (point or area) designatedon the heat map 611, which is different from the heat map 611, issuperimposed on the three-dimensional image 644. Note also that thefunction of the three-dimensional image 644 is not limited to display athree-dimensional image of the brain viewed from a desired point of thebrain. For example, as illustrated in FIG. 40, the three-dimensionalimage 644 may be a cut-model image obtained by extracting a partialimage of the brain in three-dimensional directions around the positionof the brain specified in the three-view image 613.

The peak that is selected from among the peaks registered in the peaklist 614 is identified on the three-view head image 613 as illustratedin FIG. 39, and as illustrated in FIG. 41, a peak point 646 thatindicates the above-selected peak may be displayed on thethree-dimensional image 644. For example, the top N peak positions withreference to the peak selected from the peak list 614 may be displayedon the three-dimensional image 644. FIG. 42 is a diagram illustrating anexample in which the positions of the top three peaks (i.e., the peakpoints 646, 646 a, and 646 b) are indicated, according to the presentembodiment. Alternatively, the peaks at times before and after the peakselected from the peak list 614 may be displayed in FIG. 42 as the peakpoints 646, 646 a, and 646 b in place of the above top three peaks. Inother words, the track of the peaks may be displayed. How the peakpositions are to be indicated may be determined based on the settings.For example, in addition to or in place of the above setting, thesettings may be switched between the setting in which no peak is to beindicated or the setting in which peaks whose signal strength is equalto or higher than M are indicated.

As illustrated in FIG. 43, the display mode of the multiple peaksdisplayed on the three-dimensional image 644 may be changed according tothe attribute information of those peaks. FIG. 43 is a diagramillustrating an example in which the colors of the indicated peaks arechanged so as to be different from each other, according to the presentembodiment.

As illustrated in FIG. 44, the sectional-view control unit 213 maycontrol the display to superimpose a dipole 647 that is obtained as aresult of dipole estimation on the three-dimensional image 644, in, forexample, a different analyzing screen. Due to such a configuration, therelative positions of the heat map on the three-dimensional image 644that indicates sites to be conserved and the dipole that indicates theaffected sites (target sites) can be figured out, and such informationcan be used for, for example, surgery.

On one of the sectional views of the three-view head image 613, adesired point of the brain in the three-dimensional space can bespecified by a clicking or tapping operation performed on the input unit208 by the analyst. Once a particular position is specified on thethree-view images as described above, the distribution of the signalstrength of the biomedical signals of the time and frequencycorresponding to the specified position is reflected in the heat map611.

On one of the sectional views of the three-view head image 613, aspecific area of the brain in the three-dimensional space can bedesignated by a dragging operation or swiping operation made by theanalyst to the input unit 208. Once a desired area is specified on thethree-view images as described above, the distribution of the averagesignal strength of the biomedical signals of the time and frequencycorresponding to the specified area is reflected in the heat map 611.

In the heat map that is drawn on the three-dimensional image 644 (andthe three-view images), which is a contour map that indicates thedifferences in signal strength, results of stimulation such asactivation at varying sites of the brain may be superimposed on top ofone another. For example, after a result of performing languagestimulation during the measurement and a result of performing visualstimulation during the measurement are obtained, as illustrated in FIG.45C, the sectional-view control unit 213 may superimpose a heat mapwhere the language area is activated as illustrated in FIG. 45A and aheat map where the visual area is activated as illustrated in FIG. 45Bon top of one another. Due to such a configuration as above, it becomesidentifiable as illustrated in FIG. 45C that the sites indicated on theheat map where superimposition has been performed are sites to beconserved. Such superimposition may be implemented as follows. Assumingthat the currently-displayed result of measurement indicates thelanguage area, it may be configured such that a different result ofmeasurement (for example, the result of measurement indicating thevisual area) is selectable from a menu. When superimposition is to beperformed, the reaction time to the stimulation may vary depending onthe object. In view of such circumstances, if the time lag isconfigurable when an object is added, superimposition can be performedmore precisely. Further, the three-dimensional image as illustrated inFIG. 45C, which is obtained as a result of superimposing a heat map on athree-dimensional image of the brain, may be highlighted in an inversemanner as illustrated in FIG. 45D. Due to this configuration, aremovable site, which is not among the sites to be conserved, can beindicated in the reversed manner.

In the present embodiment, the sectional view in the three-view headimage 613 includes three cross sections taken from three differentdirections. However, no limitation is intended thereby, and thesectional view in the three-view head image 613 may be a single crosssection taken from one specific direction or two or four or more crosssections taken from different directions.

With reference to FIG. 46 to FIG. 48, operations in the time-frequencyanalysis screen 601 are described in which the changes in viewpoint madeon the three-dimensional image 644 of the three-view head image 613 arereflected in the three-dimensional images of the three-dimensional view612.

FIG. 46 is a diagram illustrating a state before the viewpoint ischanged for the three-dimensional image 644 of the three-view head image613, according to the present embodiment.

FIG. 47 is a diagram illustrating a dialog box displayed when theviewpoint of the three-dimensional image 644 of the three-view headimage 613 is changed, according to the present embodiment.

FIG. 48 is a diagram illustrating a setting in which the changes inviewpoint made on the three-dimensional image 644 are applied to theviewpoint of the three-dimensional images in the first row of thethree-dimensional view 612, according to the present embodiment.

FIG. 49 is a diagram illustrating a state in which the changes inviewpoint of the three-dimensional image 644 in the three-view headimage 613 are applied to the viewpoint of the three-dimensional imagesin the first row of the three-dimensional view 612, according to thepresent embodiment.

FIG. 50 is a diagram illustrating a setting in which the changes inviewpoint made on the three-dimensional image 644 are reflected in thethree-dimensional images in the first and second rows of thethree-dimensional view 612, according to the present embodiment.

FIG. 51 is a diagram illustrating a state in which the changes in theviewpoint of the three-dimensional image 644 of the three-view headimage 613 are reflected in the first and second rows of thethree-dimensional view 612, according to the present embodiment.

FIG. 52 is a diagram illustrating a setting in which the changes inviewpoint made on the three-dimensional image 644 are symmetricallyreflected in the three-dimensional images in the first and second rowsof the three-dimensional view 612, according to the present embodiment.

FIG. 46 is a diagram illustrating a state in which the changes in theviewpoint of the three-dimensional image 644 of the three-view headimage 613 are symmetrically reflected in the three-dimensional images inthe first and second rows of the three-dimensional view 612, accordingto the present embodiment.

FIG. 47 is a diagram illustrating a setting in which newthree-dimensional images in which the changes in viewpoint made on thethree-dimensional image 644 are reflected are added to thethree-dimensional view 612 in a separate row, according to the presentembodiment.

FIG. 48 is a diagram illustrating a state in which new three-dimensionalimages in which the changes in viewpoint made on the three-dimensionalimage 644 of the three-view head image 613 are reflected are added tothe three-dimensional view 612 in a separate row, according to thepresent embodiment.

In a similar manner to the three-dimensional view 612, the viewpoint ofthe image of the brain displayed on the three-dimensional image 644 ofthe three-view head image 613 can be changed as manipulated by theanalyst (for example, a dragging operation or a swiping operation). Insuch cases, the changes in the viewpoint of the brain in thethree-dimensional image 644 may be reflected in the viewpoint of thethree-dimensional images of the brain displayed in the three-dimensionalview 612. Some patterns of reflection methods or application methods aredescribed below.

Once the three-dimensional image 644 of the three-view head image 613displayed on the time-frequency analysis screen 601, as illustrated inFIG. 46, is manipulated by the analyst (for example, a draggingoperation or a swiping operation), the sectional-view control unit 213controls the display to display the dialog box 650 as illustrated inFIG. 47. The dialog box 650 appears when the viewpoint of the brain inthe three-dimensional image 644 is changed, and is a window used todetermine how such changes in viewpoint are to be reflected in thethree-dimensional view 612. For example, when a key “Do not make changesin three-dimensional view” is clicked or tapped in the presentembodiment, the viewpoint of the three-dimensional images in thethree-dimensional view 612 is not changed. In the present embodiment, asillustrated in FIG. 47, it is assumed that the analyst changes theviewpoint of the three-dimensional image 644 viewed from the viewpointon the left side of the brain so as to display the three-dimensionalimages of the brain viewed from a rear side.

Firstly, as illustrated in FIG. 48, a case is described in which the key“Reflect changes in row of three-dimensional view” in the dialog box 650is clicked or tapped. In response to this operation, the sectional-viewcontrol unit 213 controls the display to display a dialog box 651 asillustrated in FIG. 48 to determine how such changes in viewpoint are tobe reflected in the three-dimensional view 612. As illustrated in FIG.48, the analyst selects the first row of the three-dimensional view 612as the row in which changes are to be reflected and then the selects“Apply same viewpoint to three-dimensional images” in the dialog box651. In such a case, as illustrated in FIG. 49, the three-dimensionaldisplay control unit 212 controls the display to display thethree-dimensional images in the first row (upper row) of thethree-dimensional view 612 to have the viewpoint same as the changedviewpoint of the three-dimensional image 644.

Next, a case is described in which, after the viewpoint is changed asillustrated in FIG. 47, the analyst clicks or taps the key “Reflectchanges in row of three-dimensional view” in the dialog box 650 asillustrated in FIG. 50, and the analyst selects the first and secondrows of the three-dimensional view 612 as the row in which changes areto be reflected and then selects “Change viewpoints of three-dimensionalimages accordingly” in the dialog box 651. In such a case, asillustrated in FIG. 51, the three-dimensional display control unit 212controls the display to reflect the changes in viewpoint made on thethree-dimensional image 644 in the three-dimensional images in the firstrow of the three-dimensional view 612, which originally have the sameviewpoint as the three-dimensional image 644. In other words, asillustrated in FIG. 51, changes are reflected so as to display thethree-dimensional images of the brain viewed from a rear side. Further,as illustrated in FIG. 51, the three-dimensional display control unit212 controls the display to reflect the changes in viewpoint made on thethree-dimensional image 644 in the three-dimensional images in thesecond row of the three-dimensional view 612, which originally have theviewpoint on the right side of the brain. In other words, as illustratedin FIG. 51, the viewpoint is changed so as to display thethree-dimensional images of the brain viewed from a front side. Notealso that the selection made in the dialog box 651 by clicking ortapping the key “Reflect changes in row of three-dimensional view” maybe set to the initial state. Upon that selection, for example, a “Viewlink” or “Release view link” key may be arranged to display the resultof selection. Due to such a configuration, repetitive selectingoperation can be omitted or simplified.

Next, as illustrated in FIG. 53, a case is described in which theviewpoint of the three-dimensional image 644 viewed from the viewpointon the left side of the brain is changed by the analyst so as to displaythe three-dimensional images of the brain viewed from a left-frontalside. Upon these changes, as illustrated in FIG. 52, the analyst clicksor taps the key “Reflect changes in row of three-dimensional view” inthe dialog box 650, and then selects the first and second rows of thethree-dimensional view 612 as the row in which changes are to bereflected and selects “Change viewpoints of three-dimensional imagessymmetrically” in the dialog box 651. In such a case, as illustrated inFIG. 53, the three-dimensional display control unit 212 controls thedisplay to reflect the changes in viewpoint made on thethree-dimensional image 644 in the three-dimensional images in the firstrow of the three-dimensional view 612, which originally have the sameviewpoint as the three-dimensional image 644. In other words, asillustrated in FIG. 53, the viewpoint is changed so as to display thethree-dimensional images of the brain viewed from a left-frontal side ofthe brain. Further, as illustrated in FIG. 53, the three-dimensionaldisplay control unit 212 controls the display to symmetrically reflectthe changes in viewpoint made on the three-dimensional image 644 in thethree-dimensional images in the second row of the three-dimensional view612, which originally have the viewpoint on the right side of the brain.As illustrated in FIG. 53, the viewpoint of the three-dimensional imagesin the second row of the three-dimensional view 612 is changed to besymmetrical to the center plane of the brain (symmetry plane). In otherwords, the viewpoint of the three-dimensional images in the second rowof the three-dimensional view 612 is changed so as to display thethree-dimensional images of the brain viewed from a right-frontal sideof the brain.

Next, a case is described in which, after the viewpoint is changed asillustrated in FIG. 54, the analyst selects “Apply same viewpoint tothree-dimensional images” in a dialog box 652 displayed by clicking ortapping the key “Add new row in three-dimensional view” in the dialogbox 650 as illustrated in FIG. 54. In such a case, as illustrated inFIG. 55, as a result of the changes in viewpoint made on thethree-dimensional image 644, the three-dimensional display control unit212 controls the display to add the three-dimensional images of thebrain with the same viewpoint as a new row in a display area 612-3 ofthe three-dimensional view 612. In other words, as illustrated in FIG.55, three-dimensional images of the brain viewed from a rear side aredisplayed in a new row of the display area 612-3.

As described above, in accordance with the various kinds of settings,the changes in viewpoint made on the three-dimensional image 644 in thethree-view head image 613 can be reflected in the viewpoint of thethree-dimensional images of the brain that are arranged in thethree-dimensional view 612 in a chronological order. Due to such aconfiguration, changes in viewpoint similar to the changes in viewpointmade on the three-dimensional image 644 do not have to be made on thethree-dimensional view 612 in a repetitive manner. Due to thisconfiguration, the operability or efficiency improves. Furthermore, thechanges in the state of the brain can be checked on thethree-dimensional view 612 in chronological order with the viewpointsame as the viewpoint as changed in the three-dimensional image 644 orwith the viewpoint corresponding to the viewpoint as changed in thethree-dimensional image 644.

The above methods of reflecting changes in viewpoint in each one of theimages of the brain displayed in the three-dimensional view 612, whichare set or determined in the dialog boxes 650 to 652 as illustrated inFIG. 48, FIG. 50, FIG. 52, and FIG. 54, are given by way of example, andany other ways or methods of reflection may be set or adopted.

Operations in which changes in viewpoint are reflected in thethree-dimensional images of the three-dimensional view 612 when theviewpoint of the brain in the three-dimensional image 644 is changed aredescribed as above. However, no limitation is indicated thereby, and thedisplay mode that is to be changed for the three-dimensional image 644is not limited to viewpoint. For example, the display mode that is to bechanged for the three-dimensional image 644 may be, for example, changesin size, changes in brightness, or changes in transparency. Such changesmay be reflected in the three-dimensional images of thethree-dimensional view 612 without departing from the spirit or scope ofthe disclosure of the above changes in viewpoint.

Basic operation of the peak list 614 on the time-frequency analysisscreen 601 is described below with reference to FIG. 56 to FIG. 60.

FIG. 56 is a diagram illustrating the setting of the peak list 614,according to the present embodiment.

FIG. 57 is a diagram illustrating a spatial peak according to thepresent embodiment.

FIG. 58 is a diagram illustrating a peak in time and a peak in frequencyaccording to the present embodiment.

FIG. 59 is a diagram illustrating how a specific peak is selected fromthe drop-down peak list 614, according to the present embodiment.

FIG. 60 is a diagram illustrating a state in which the peak selectedfrom the drop-down peak list 614 is reflected in the heat map 611, thethree-dimensional view 612, and the three-view head image 613, accordingto the present embodiment.

In the peak list 614, the peaks in signal strength that meet a specifiedcondition, which are extracted by the peak list controller 203, areregistered. As illustrated in FIG. 56, the peak-list controller 203controls the display to display a pull-down list 656, indicating a listof signal strengths registered as the peak list 614 is pulled down.

The above conditions for a peak in regard to the signal strength, whichis extracted by the peak-list controller 203, can be configured byclicking or tapping the peak-list setting key 614 a. Once the peak-listsetting key 614 a is clicked or tapped, the peak-list controller 203controls the display to display a dialog box 655 where the conditionsfor a peak in regard to the extracted signal strength can be configured.

In the dialog box 655, first of all, how the peak data registered in thepeak list 614 is to be sorted can be configured. When “Sort values ofpeaks in descending order” is selected in a dialog box 655, thepeak-list controller 203 sorts the peaks of the signal strength in thepeak data registered in the peak list 614 in descending order. Bycontrast, when “Sort levels of peaks (difference in height between topand bottom) in descending order” is selected in the dialog box 655, thepeak-list controller 203 sorts the peak data registered in the peak list614 in descending order of the difference between the signal strength atthe peak point and the signal strength at the bottom of the peak.

Further, in the dialog box 655, what type of peak data is to beregistered (listed) in the peak list 614 can be configured. When “Allspatial peaks” is selected in the dialog box 655, the peak-listcontroller 203 extracts the spatial peaks, in the entirety of the brain,at each time and each frequency on the plane of time and frequency, andregisters the extracted spatial peaks in the peak list 614. The term“spatial peaks” in the present embodiment indicates the peaks of signalstrength of a biomedical signal of the time and frequency of interest inthe entirety of the brain, and the signal strength in the peak spot 801is greater than that of the area around that peak spot, like a peak spot801 as illustrated in FIG. 57.

When “All peaks in time/frequency” is selected in the dialog box 655,the peak-list controller 203 extracts all the peaks in time andfrequency from varying points of the plane of time and frequency in theentirety of the brain and registers the extracted peaks in the peak list614. The term “peaks in time and frequency” in the present embodimentindicate the peaks of signal strength of a biomedical signal at a siteof interest in the brain on the plane of time and frequency, like a peakspot 802 as illustrated in FIG. 58, and the signal strength in the peakspot 802 is greater than that of the area around that peak spot.

When “Spatial peaks in designated time/frequency” is selected in thedialog box 655, the peak-list controller 203 extracts the spatial peaksat the time and frequency specified on the plane of time and frequencyin the entirety of the brain, and registers the extracted spatial peaksin the peak list 614. Note also that the specified time and frequency isnot limited to a point, and the time and frequency may be selected orspecified by an area or range.

When “Peaks in time/frequency at designated position” is selected in thedialog box 655, the peak-list controller 203 extracts all the peaks intime and frequency on the plane of time and frequency at the specifiedsite of the brain, and registers the extracted peaks in the peak list614. Note also that the designated position is not limited to a point,and the position may be selected or specified by an area or range. Forexample, when a peak on a visual area is to be extracted, the entiretyof the occipital region of the brain may be specified. By so doing, apeak can easily be extracted as desired.

Next, operations are described that are performed when a specific itemof peak data is selected from the peak list 614 in which several itemsof peak data are registered. When a specific item of peak data (forexample, “95%/9 ms/70 Hz, voxel: 1736” as depicted in FIG. 59) isselected by the analyst from the pull-down list 656 of the peak list614, the heat-map display control unit 211 controls the display todisplay the heat map 611 that corresponds to a desired point of thebrain indicated by the selected item of peak data. In such aconfiguration, as described above with reference to FIG. 14, theheat-map display control unit 211 may specifically indicate on the heatmap 611 the peak that is indicated by the selected item of peak data.

Moreover, the three-dimensional display control unit 212 controls thedisplay to display the three-dimensional image of the brain of the timeand frequency that the selected item of peak data indicates in thecenter of each row of the three-dimensional view 612, and furthercontrols the display to display the three-dimensional images of thebrain at times before and after the time indicated by the selected peakdata in the three-dimensional view 612. In such cases, the heat mapsthat are superimposed on the multiple three-dimensional images of thebrain in the three-dimensional view 612 may correspond to the signalstrength of the biomedical signal with the frequency that the selecteditem of peak data indicates.

The sectional-view control unit 213 controls the display to displaythree-view images that go through the position of the brain indicated bythe selected item of peak data, in the three-view head image 613.Further, the sectional-view control unit 213 may control the display tosuperimpose the heat map, which corresponds to the signal strength ofthe biomedical signal with the time and frequency that the selected itemof peak data indicates, on the image of the brain in thethree-dimensional image 644. As illustrated in FIG. 60, thesectional-view control unit 213 may controls the display to display acut-model image, which is obtained by extracting a partial image of thebrain in three-dimensional directions around the position of the brainindicated by the selected peak data, on the three-dimensional image 644.

As described above, a specific item of peak data is selected from thepeak data registered in the peak list 614, and the heat map 611, thethree-dimensional view 612, and the three-view head image 613 thatcorrespond to the selected item of peak data are displayed accordingly.Due to such a configuration, to what position, time, and frequency ofthe brain the selected peak belongs can instantly be recognized.Further, the states of signal strength at the selected peak and at thetime and frequency around the selected peak can be figured out, and thestates of signal strength on the brain at the peak and around the peakcan also be figured out on the heat map 611.

The operations when the replay control panel 615 on the time-frequencyanalysis screen 601 is manipulated are described below with reference toFIG. 61A to FIG. 63B.

FIG. 61A and FIG. 61B are diagrams illustrating how the viewing of theheat map 611 and the three-dimensional view 612 are viewed by operationson the replay control panel 615, according to the present embodiment.

FIG. 62A and FIG. 62B are diagrams illustrating how the viewing of theheat map 611 and the three-dimensional view 612 are returned on aframe-by-frame basis by operations on the replay control panel 615,according to the present embodiment.

FIG. 63A and FIG. 63B are diagrams illustrating how the heat map 611 andthe three-dimensional view 612 are advanced on a frame-by-frame basis byoperations on the replay control panel 615, according to the presentembodiment.

The replay control panel 615 is a user interface manipulated by theanalyst to view the states of the heat map 611, the three-dimensionalview 612, and the three-view head image 613 as time elapses.

For example, when the analyst touches or clicks the “replay” key on thereplay control panel 615, as illustrated in FIG. 61A and FIG. 61B, theviewing control unit 214 instructs the heat-map display control unit 211to move the specified area 622-1 specified on the heat map 611 and therelated areas 622-2 to 622-5 around the specified area 622-1 in theright direction (i.e., the direction in which the time advances) as timeelapses. As the specified area 622-1 and the related areas 622-2 to622-5 are moved on the heat map 611, the viewing control unit 214instructs the three-dimensional display control unit 212 to switch tothe display of the three-dimensional images of the brain that correspondto the relevant multiple areas, as illustrated in FIG. 61A and FIG. 61B.As the specified area 622-1 is moved on the heat map 611 moves, theviewing control unit 214 instructs the sectional-view control unit 213to display the heat map of the signal strength that corresponds to therange of time and frequency of the moving specified area 622-1 on thethree-view images and the three-dimensional image 644.

When the analyst touches or clicks the “Frame-by-frame return” key onthe replay control panel 615, as illustrated in FIG. 62A and FIG. 62B,the viewing control unit 214 instructs the heat-map display control unit211 to move the specified area 622-1 specified on the heat map 611 andthe related areas 622-2 to 622-5 around the specified area 622-1 in theleft direction (i.e., the direction in which the time returns) by acertain length of time. As the specified area 622-1 and the relatedareas 622-2 to 622-5 are moved on the heat map 611, the viewing controlunit 214 instructs the three-dimensional display control unit 212 toswitch to the display of the three-dimensional images of the brain thatcorrespond to the relevant multiple areas, as illustrated in FIG. 62Aand FIG. 62B. As the specified area 622-1 is moved on the heat map 611moves, the viewing control unit 214 instructs the sectional-view controlunit 213 to display the heat map of the signal strength that correspondsto the range of time and frequency of the moved specified area 622-1 onthe three-view images and the three-dimensional image 644.

When the analyst touches or clicks the “frame-by-frame advance” key onthe replay control panel 615, as illustrated in FIG. 63A and FIG. 63B,the viewing control unit 214 instructs the heat-map display control unit211 to move the specified area 622-1 specified on the heat map 611 andthe related areas 622-2 to 622-5 around the specified area 622-1 in theright direction (i.e., the direction in which the time advances) by acertain length of time. As the specified area 622-1 and the relatedareas 622-2 to 622-5 are moved on the heat map 611, the viewing controlunit 214 instructs the three-dimensional display control unit 212 toswitch to the display of the three-dimensional images of the brain thatcorrespond to the relevant multiple areas, as illustrated in FIG. 63Aand FIG. 63B. As the specified area 622-1 is moved on the heat map 611moves, the viewing control unit 214 instructs the sectional-view controlunit 213 to display the heat map of the signal strength that correspondsto the range of time and frequency of the moved specified area 622-1 onthe three-view images and the three-dimensional image 644.

When the analyst touches or clicks the “stop” key on the replay controlpanel 615, the viewing control unit 214 instructs each one of theheat-map display control unit 211, the three-dimensional display controlunit 212, and the sectional-view control unit 213 to terminate itsdisplay operation on the heat map 611, the three-dimensional view 612,and the three-view head image 613.

When the analyst touches or clicks the “move to head” key on the replaycontrol panel 615, the viewing control unit 214 instructs the heat-mapdisplay control unit 211 to move the specified area 622-1 specified onthe heat map 611 to the head of the time. As the specified area 622-1 ismoved on the heat map 611 moves, the viewing control unit 214 instructsthe three-dimensional display control unit 212 to switch to the displayof the three-dimensional images of the brain that correspond to thespecified area 622-1. As the specified area 622-1 is moved on the heatmap 611 moves, the viewing control unit 214 instructs the sectional-viewcontrol unit 213 to display the heat map of the signal strength thatcorresponds to the range of time and frequency of the moved specifiedarea 622-1 on the three-view images and the three-dimensional image 644.

When the analyst touches or clicks the “move to end” key on the replaycontrol panel 615, the viewing control unit 214 instructs the heat-mapdisplay control unit 211 to move the specified area 622-1 specified onthe heat map 611 to the end of the time. As the specified area 622-1 ismoved on the heat map 611 moves, the viewing control unit 214 instructsthe three-dimensional display control unit 212 to switch to the displayof the three-dimensional images of the brain that correspond to thespecified area 622-1. As the specified area 622-1 is moved on the heatmap 611 moves, the viewing control unit 214 instructs the sectional-viewcontrol unit 213 to display the heat map of the signal strength thatcorresponds to the range of time and frequency of the moved specifiedarea 622-1 on the three-view images and the three-dimensional image 644.

Due to the viewing and displaying operation as described above, thechanges over time in the distribution (heat map) of the signal strengthindicated on the three-view head image 613 and the three-dimensionalview 612 can be checked in moving images, and for example, the movementof the peaks over time can visually be checked.

How the heat map 611, the three-dimensional view 612, and the three-viewhead image 613 are initially displayed when the time-frequency analysisscreen 601 is started (opened) according to the present embodiment aredescribed below with reference to FIG. 64 to FIG. 66.

FIG. 64 is a diagram illustrating from what viewpoint the images are tobe initially displayed with respect to a peak, according to the presentembodiment.

FIG. 65 is a diagram illustrating from what viewpoint the images are tobe initially displayed with respect to a pair of peaks, according to thepresent embodiment.

FIG. 66 is a diagram illustrating a state in which the images of thebrain viewed from the viewpoints as illustrated in FIG. 65 are displayedin the three-dimensional view 612 as the initial display.

Some patterns of what kind of images are to be initially displayed asthe heat map 611, the three-dimensional view 612, and the three-viewhead image 613 when the analyst started (opened) the time-frequencyanalysis screen 601 are described below.

For example, the analysis display controller 202 calculates the time andfrequency and the position inside the brain where the signal strength ismaximized throughout the entire range of time and frequency in theentirety of the brain. In such a case, the heat-map display control unit211 controls the display to display the heat map 611 at the positioninside the brain calculated by the analysis display controller 202.Moreover, the three-dimensional display control unit 212 controls thedisplay to display the three-dimensional image of the brain, whichcorresponds to the time and frequency calculated by the analysis displaycontroller 202, where the signal strength is maximized, in thethree-dimensional view 612. The sectional-view control unit 213 controlsthe display to display three-view images that go through the positioninside the brain calculated by the analysis display controller 202 inthe three-view head image 613, and superimposes the heat map of time andfrequency calculated by the analysis display controller 202, where thesignal strength is maximized, on the three-view images and thethree-dimensional image 644.

The analysis display controller 202 may calculate the position insidethe brain where the average of signal strength is maximized throughoutthe entire range of time and frequency. In such a case, the heat-mapdisplay control unit 211 controls the display to display the heat map611 at the position inside the brain calculated by the analysis displaycontroller 202. Moreover, the three-dimensional display control unit 212controls the display to display the three-dimensional image of thebrain, which corresponds to the time and frequency on the displayed heatmap 611 where the signal strength is maximized, on the three-dimensionalview 612. The sectional-view control unit 213 controls the display todisplay three-view images that go through the position inside the braincalculated by the analysis display controller 202 in the three-view headimage 613, and superimposes the heat map of the time and frequency,where the signal strength is maximized in the displayed heat map 611, onthe three-view images and the three-dimensional image 644.

Alternatively, the analysis display controller 202 may compute andobtain the time and frequency where the average of the signal strengthis maximized in the entirety of the brain. In such a case, thethree-dimensional display control unit 212 controls the display todisplay the three-dimensional image of the brain that corresponds to thetime and frequency calculated by the analysis display controller 202 onthe three-dimensional view 612. The heat-map display control unit 211computes and obtains the position inside the brain, which is displayedon the three-dimensional images of the three-dimensional view 612, wherethe signal strength is maximized in the heat map that corresponds to thetime and frequency calculated by the analysis display controller 202,and controls the display to display the heat map 611 at the computed andobtained position. The sectional-view control unit 213 controls thedisplay to display three-view images that go through the position insidethe brain calculated by the heat-map display control unit 211 in thethree-view head image 613, and superimposes the heat map of the time andfrequency calculated by the analysis display controller 202 on thethree-view images and the three-dimensional image 644.

Moreover, the three-dimensional display control unit 212 may control thedisplay to display the heat map 611 at a position inside the brainindicated by the first item of peak data in the peak data registered inthe peak list 614. Moreover, the three-dimensional display control unit212 controls the display to display the three-dimensional image of thebrain, which corresponds to the time and frequency indicated by thefirst item of peak data in the peak data registered in the peak list614, on the three-dimensional view 612. The sectional-view control unit213 controls the display to display three-view images that go throughthe position inside the brain indicated by the first item of peak datain the peak data registered in the peak list 614, in the three-view headimage 613, and superimposes the heat map of the time and frequencyindicated by the selected item of peak data, on the three-view imagesand the three-dimensional image 644.

Moreover, the three-dimensional display control unit 212 may control thedisplay to display the heat map 611 at the position inside the brainthat is preset depending on an object to be measured (for example, avisual area, auditory area, somatosensory area, motor area, and alanguage area are the preset parameters). Moreover, thethree-dimensional display control unit 212 controls the display todisplay the three-dimensional image of the brain, which corresponds tothe time and frequency that is preset depending on an object to bemeasured (for example, a visual area, auditory area, somatosensory area,motor area, and a language area), on the three-dimensional view 612. Thesectional-view control unit 213 controls the display to displaythree-view images that are preset depending on an object to be measured(for example, a visual area, auditory area, somatosensory area, motorarea, and a language area) and go through the position inside the brainin the three-view head image 613, and superimposes the heat map of thetime and frequency indicated by the selected item of peak data, on thethree-view images and the three-dimensional image 644.

The initial viewpoint of the three-dimensional images of the brain inthe three-dimensional view 612 and the three-dimensional image 644 onthe three-view head image 613 that are displayed when the analyststarted (opened) the time-frequency analysis screen 601 is describedbelow.

For example, the viewpoint that is preset depending on an object to bemeasured (for example, a visual area, auditory area, somatosensory area,motor area, and a language area) may be employed for the initialviewpoint. In such a configuration, the number of rows (viewpoints) inthe three-dimensional view 612 is also preset in advance. For example,when the three-dimensional view 612 consists of two rows, two viewpointsneed to be preset in advance. For example, when the language areas areto be measured, the viewpoints on the right and left sides of the brainare preset in advance.

A viewpoint from which the peak that is registered in the forefront ofthe peak list 614 can be observed most clearly may be employed for theinitial viewpoint. More specifically as illustrated in FIG. 64, aviewpoint P0 may be set on a straight line 811 that connects the centerof the brain and a peak, as the initial viewpoint.

A viewpoint that is determined based on a peak whose predeterminedparameter in the peak list 614 (for example, the value of the peak(signal strength) or the level of the peak as illustrated in FIG. 57)exceeds a prescribed threshold may be employed for the initialviewpoint. For example, when there are two peaks that have exceeded thethreshold, the three-dimensional view 612 may be displayed in two rows,and as illustrated in FIG. 65, viewpoints P1 and P2 may be set as theinitial viewpoints on straight lines 812 and 813 that connect the centerof the brain and the respective peaks. An example of such aconfiguration as above is illustrated in FIG. 66 in which thethree-dimensional images of the brain viewed from the viewpoint P1 aredisplayed in the upper row of the three-dimensional view 612 and thethree-dimensional images of the brain viewed from the viewpoint P2 aredisplayed in the lower row of the three-dimensional view 612.

As described above, in the present embodiment, particular time andfrequency are specified on the heat map 611 that indicates thedistribution of the signal strength of biomedical signals, and athreshold is determined based on the value of the signal strength at thespecified position. Due to this configuration, the area of the time andfrequency that has signal strength meeting a prescribed condition isautomatically extracted. Accordingly, a concrete area of biomedicalsignals of interest at particular time and frequency can easily beextracted. As such a concrete area can be specified in an automaticmanner, a more appropriate heat map can be superimposed on thethree-view head image 613 and the three-dimensional view 612.

Moreover, in the above embodiment of the present disclosure, the heatmap 611 indicating the time and frequency of a biomedical signal at aspecific site of the brain or in a specific area of the brain isdisplayed. Moreover, the three-dimensional images indicative of theactivity of the brain at times before and after the above time aredisplayed around the three-dimensional image on which a heat mapindicative of the activity of the brain at the point designated on theheat map 611 or in the area designated on the heat map 611 issuperimposed. In other words, some still images (i.e., three-dimensionalimages) that indicate the activity of the brain are advanced or returnedon a frame-by-frame basis in the above embodiment of the presentdisclosure. Due to this configuration, still images that indicate theactivity of the brain can appropriately and promptly be extracted, andthe activity of the brain can easily be analyzed. Further, a conferenceor discussion can take place based on those images in an effectivemanner.

In the above embodiment of the present disclosure, once a specific itemof peak data is selected from the peak data registered in the peak list614, the heat map 611, the three-dimensional view 612, and thethree-view head image 613 that correspond to the selected item of peakdata are displayed. Due to such a configuration, to what position, time,and frequency of the brain the selected peak belongs can instantly berecognized. Further, the states of signal strength at the selected peakand at the time and frequency around the selected peak can be figuredout, and the states of signal strength on the brain at the peak andaround the peak can also be figured out on the heat map 611.

The viewpoint of the brain can be changed as desired in thethree-dimensional view 612, and the changes based on the changedviewpoint of the brain can be reflected in the images of the brain inthe same row or in a different row. Due to this configuration, thechanges that are made in the viewpoint of a specific three-dimensionalimage (i.e., the target three-dimensional image) are automaticallyreflected in the other three-dimensional images, and the operability orefficiency improves. Furthermore, the images of the brain in multiplerows can be compared with each other, and thus the changes in activityamong the images of the brain that are viewed from a correspondingviewpoint and are temporally close to each other can easily be checked.As the viewpoint of the brain that is drawn as three-dimensional imagescan be changed as desired, a firing point that cannot be viewed from oneviewpoint can be checked.

As described above, in accordance with the various kinds of settings,the changes in viewpoint made on the three-dimensional image 644 in thethree-view head image 613 can be reflected in the viewpoint of thethree-dimensional images of the brain that are arranged in thethree-dimensional view 612 in a chronological order. Due to such aconfiguration, changes in viewpoint similar to the changes in viewpointmade on the three-dimensional image 644 do not have to be made on thethree-dimensional view 612 in a repetitive manner. Accordingly, theoperability or efficiency improves. Furthermore, the changes in thestate of the brain can be checked on the three-dimensional view 612 inchronological order with the viewpoint same as the viewpoint as changedin the three-dimensional image 644 or with the viewpoint correspondingto the viewpoint as changed in the three-dimensional image 644.

In the above embodiment of the present disclosure, a biomedical signalof the brain, which is an example of a biological site, is considered.However, no limitation is intended thereby, and it can be applied to thebiomedical signals of a biological site such as a spinal cord andmuscles. For example, in the case of lumber spine (lumbar vertebra), thethree-dimensional view 612 that is used as an image of the brain may bedisplayed as illustrated in FIG. 67A, FIG. 67B, FIG. 67C, and FIG. 67D.FIG. 67A, FIG. 67B, FIG. 67C, and FIG. 67D illustrates how a lumbarsignal is transmitted to the upper side in chronological order.

In the above embodiment of the present disclosure and its modifications,when at least some of the multiple functional units of thebiomedical-signal measuring system 1 is implemented by executing aprogram, such a program may be incorporated in advance in a read onlymemory (ROM) or the like. The program to be executed by thebiomedical-signal measuring system 1 according to the above embodimentof the present disclosure and its modifications may be installed fordistribution in any desired computer-readable recording medium such as acompact disc, a read-only memory (CD-ROM), a flexible disk (FD), acompact disc-recordable (CD-R), and a digital versatile disk (DVD) in afile format installable or executable by a computer. The program that isexecuted in the biomedical-signal measuring system 1 according to theabove embodiment of the present disclosure and its modifications may beprovided upon being stored in a computer connected to a network such asthe Internet and downloaded through the network. A program to beexecuted by the biomedical-signal measuring system 1 according to theabove embodiment of the present disclosure and its modifications may beprovided or distributed through a network such as the Internet. Aprogram to be executed by the biomedical-signal measuring system 1according to the above embodiment of the present disclosure and itsmodifications has module structure including at least one of theabove-described functional units. Regarding the actual hardware relatedto the program, the CPU 101 reads and executes the program from thememory as described above (e.g., the ROM 103) to load the program ontothe main memory (e.g., the RAM 102) to implement the above multiplefunctional units.

Each of the functions of the described embodiments may be implemented byone or more processing circuits or circuitry. Processing circuitryincludes a programmed processor, as a processor includes circuitry. Aprocessing circuit also includes devices such as an application specificintegrated circuit (ASIC), digital signal processor (DSP), fieldprogrammable gate array (FPGA), and conventional circuit componentsarranged to perform the recited functions.

What is claimed is:
 1. An information processing device comprisingcircuitry to: obtain specific information of a first intensitydistribution of a biomedical signal from a source, the specificinformation being displayed on a display; and extract, based on thespecific information, a specified area of the first intensitydistribution to obtain second intensity distribution of the biomedicalsignal to be superimposed on an image indicative of the source.
 2. Theinformation processing device according to claim 1, wherein the firstintensity distribution is a two-dimensional distribution of thebiomedical signal with respect to time and frequency, and wherein thecircuitry obtains at least a particular time and frequency of thetwo-dimensional distribution as the specific information.
 3. Theinformation processing device according to claim 2, wherein the firstintensity distribution is a two-dimensional distribution of signalstrength of the biomedical signal with respect to time and frequency,and wherein the circuitry further obtains, as the specific information,a value of the signal strength at the particular time and frequency. 4.The information processing device according to claim 2, wherein thefirst intensity distribution is a two-dimensional distribution ofplausibility, distribution, or a standard deviation of signal strengthof the biomedical signal, with respect to time and frequency, andwherein the circuitry further obtains, as the specific information, avalue of plausibility, distribution, or a standard deviation of thesignal strength at the particular time and frequency.
 5. The informationprocessing device according to claim 3, wherein the circuitry performsbinarization on the first intensity distribution, based on the valuethat serves as the specific information, to obtain a binarized image,wherein the circuitry extracts, as the specified area, a combined areain the binarized image, the combined area including a point at theparticular time and frequency.
 6. The information processing deviceaccording to claim 5, wherein the circuitry uses the value that servesas the specific information as a threshold, and makes a determination ona value of each pixel in the first intensity distribution to obtain thebinarized image.
 7. The information processing device according to claim5, wherein the circuitry uses, as a threshold, a value obtained bymultiplying the value that serves as the specific information by aprescribed rate, and makes a determination on a value of each pixel inthe first intensity distribution to obtain the binarized image.
 8. Theinformation processing device according to claim 1, wherein thecircuitry obtains the specific information at a position in the firstintensity distribution specified by operation or input made through aninput unit.
 9. The information processing device according to claim 1,wherein the circuitry obtains the specific information of apredetermined area of the first intensity distribution.
 10. Theinformation processing device according to claim 9, wherein thecircuitry obtains the specific information of a representative point ofthe predetermined area in the first intensity distribution.
 11. Theinformation processing device according to claim 1, wherein thecircuitry extracts, as the specified area, a rectangular area thatcircumscribes an area of the first intensity distribution extracted toobtain the second intensity distribution, based on the specificinformation.
 12. The information processing device according to claim 1,wherein the specific information includes a plurality of items specificinformation, wherein the circuitry obtains the plurality of itemsspecific information from the first intensity distribution, wherein thespecified area includes a plurality of specified areas, wherein thecircuitry extracts the specified areas of the first intensitydistribution based on the plurality of items specific information, andwherein the second intensity distribution includes a plurality of itemsof second intensity distribution based on the plurality of specifiedareas, and wherein the circuitry controls the display to superimposeeach one of the plurality of items of second intensity distribution on aseparate image indicative of the source.
 13. The information processingdevice according to claim 1, wherein the specific information includes aplurality of items specific information, wherein the circuitry obtainsthe plurality of items specific information from the first intensitydistribution, wherein the specified area includes a plurality ofspecified areas, wherein the circuitry extracts the specified areas ofthe first intensity distribution based on the plurality of itemsspecific information, and wherein the circuitry controls the display tosuperimpose the second intensity distribution on an image indicative ofthe source, the second intensity distribution being based on thespecified area selected from the plurality of specified areas byoperation or input made through an input unit.
 14. A method ofprocessing information, the method comprising: obtaining specificinformation of a first intensity distribution of a biomedical signalfrom a prescribed source, the specific information being displayed on adisplay; and extracting, based on the specific information, a specifiedarea of the first intensity distribution to obtain second intensitydistribution of the biomedical signal to be superimposed on an imageindicative of the source.
 15. A computer-readable non-transitoryrecording medium storing a program for causing a computer to execute amethod, the method comprising: obtaining specific information of a firstintensity distribution of a biomedical signal from a prescribed source,the specific information being displayed on a display; and extracting,based on the specific information, a specified area of the firstintensity distribution to obtain second intensity distribution of thebiomedical signal to be superimposed on an image indicative of thesource.
 16. A biomedical-signal measuring system comprising: ameasurement device configured to measure at least one kind of biomedicalsignal of a test subject; and the information processing deviceaccording to claim 1.